{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Copyright 2020 Google LLC\n",
    "#\n",
    "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
    "# you may not use this file except in compliance with the License.\n",
    "# You may obtain a copy of the License at\n",
    "#\n",
    "#     https://www.apache.org/licenses/LICENSE-2.0\n",
    "#\n",
    "# Unless required by applicable law or agreed to in writing, software\n",
    "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
    "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
    "# See the License for the specific language governing permissions and\n",
    "# limitations under the License."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a target=\"_blank\" href=\"https://colab.research.google.com/github/GoogleCloudPlatform/keras-idiomatic-programmer/blob/master/books/deep-learning-design-patterns/Workshops/Novice/Deep%20Learning%20Design%20Patterns%20-%20Workshop%20-%20Chapter%20III%20-%202.ipynb\">\n",
    "<img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Deep Learning Design Patterns - Code Labs\n",
    "\n",
    "## Lab Exercise #3 - Get Familiar with Hyperparameters - Answers\n",
    "\n",
    "## Prerequistes:\n",
    "\n",
    "    1. Familiar with Python\n",
    "    2. Completed Chapter III: Training Foundation\n",
    "\n",
    "## Objectives:\n",
    "\n",
    "    1. Hand setting Epochs and Mini-Batches\n",
    "    2. Use ImageDataGenerator for batch generation.\n",
    "    3. Finding good learning rate."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Epochs and Mini-Batches\n",
    "\n",
    "In this section, we will hand-roll our own code (vs. builtin feeders) to feed the training data for training. We will need to handle the following:\n",
    "\n",
    "    1. Set a mini-batch size (128) and calculate how many batches will be in the training data.\n",
    "    1. Set the number of epochs (number of times we pass the full training data for training)\n",
    "    2. Randomly shuffle the training data on each epoch.\n",
    "    3. Iterate through the training data on batch at a time.\n",
    "    \n",
    "You fill in the blanks (replace the ??), make sure it passes the Python interpreter."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1 Batch 1\n",
      "Epoch 1 Batch 2\n",
      "Epoch 1 Batch 3\n",
      "Epoch 1 Batch 4\n",
      "Epoch 1 Batch 5\n",
      "Epoch 1 Batch 6\n",
      "Epoch 1 Batch 7\n",
      "Epoch 1 Batch 8\n",
      "Epoch 1 Batch 9\n",
      "Epoch 1 Batch 10\n",
      "Epoch 1 Batch 11\n",
      "Epoch 1 Batch 12\n",
      "Epoch 1 Batch 13\n",
      "Epoch 1 Batch 14\n",
      "Epoch 1 Batch 15\n",
      "Epoch 1 Batch 16\n",
      "Epoch 1 Batch 17\n",
      "Epoch 1 Batch 18\n",
      "Epoch 1 Batch 19\n",
      "Epoch 1 Batch 20\n",
      "Epoch 1 Batch 21\n",
      "Epoch 1 Batch 22\n",
      "Epoch 1 Batch 23\n",
      "Epoch 1 Batch 24\n",
      "Epoch 1 Batch 25\n",
      "Epoch 1 Batch 26\n",
      "Epoch 1 Batch 27\n",
      "Epoch 1 Batch 28\n",
      "Epoch 1 Batch 29\n",
      "Epoch 1 Batch 30\n",
      "Epoch 1 Batch 31\n",
      "Epoch 1 Batch 32\n",
      "Epoch 1 Batch 33\n",
      "Epoch 1 Batch 34\n",
      "Epoch 1 Batch 35\n",
      "Epoch 1 Batch 36\n",
      "Epoch 1 Batch 37\n",
      "Epoch 1 Batch 38\n",
      "Epoch 1 Batch 39\n",
      "Epoch 1 Batch 40\n",
      "Epoch 1 Batch 41\n",
      "Epoch 1 Batch 42\n",
      "Epoch 1 Batch 43\n",
      "Epoch 1 Batch 44\n",
      "Epoch 1 Batch 45\n",
      "Epoch 1 Batch 46\n",
      "Epoch 1 Batch 47\n",
      "Epoch 1 Batch 48\n",
      "Epoch 1 Batch 49\n",
      "Epoch 1 Batch 50\n",
      "Epoch 1 Batch 51\n",
      "Epoch 1 Batch 52\n",
      "Epoch 1 Batch 53\n",
      "Epoch 1 Batch 54\n",
      "Epoch 1 Batch 55\n",
      "Epoch 1 Batch 56\n",
      "Epoch 1 Batch 57\n",
      "Epoch 1 Batch 58\n",
      "Epoch 1 Batch 59\n",
      "Epoch 1 Batch 60\n",
      "Epoch 1 Batch 61\n",
      "Epoch 1 Batch 62\n",
      "Epoch 1 Batch 63\n",
      "Epoch 1 Batch 64\n",
      "Epoch 1 Batch 65\n",
      "Epoch 1 Batch 66\n",
      "Epoch 1 Batch 67\n",
      "Epoch 1 Batch 68\n",
      "Epoch 1 Batch 69\n",
      "Epoch 1 Batch 70\n",
      "Epoch 1 Batch 71\n",
      "Epoch 1 Batch 72\n",
      "Epoch 1 Batch 73\n",
      "Epoch 1 Batch 74\n",
      "Epoch 1 Batch 75\n",
      "Epoch 1 Batch 76\n",
      "Epoch 1 Batch 77\n",
      "Epoch 1 Batch 78\n",
      "Epoch 1 Batch 79\n",
      "Epoch 1 Batch 80\n",
      "Epoch 1 Batch 81\n",
      "Epoch 1 Batch 82\n",
      "Epoch 1 Batch 83\n",
      "Epoch 1 Batch 84\n",
      "Epoch 1 Batch 85\n",
      "Epoch 1 Batch 86\n",
      "Epoch 1 Batch 87\n",
      "Epoch 1 Batch 88\n",
      "Epoch 1 Batch 89\n",
      "Epoch 1 Batch 90\n",
      "Epoch 1 Batch 91\n",
      "Epoch 1 Batch 92\n",
      "Epoch 1 Batch 93\n",
      "Epoch 1 Batch 94\n",
      "Epoch 1 Batch 95\n",
      "Epoch 1 Batch 96\n",
      "Epoch 1 Batch 97\n",
      "Epoch 1 Batch 98\n",
      "Epoch 1 Batch 99\n",
      "Epoch 1 Batch 100\n",
      "Epoch 1 Batch 101\n",
      "Epoch 1 Batch 102\n",
      "Epoch 1 Batch 103\n",
      "Epoch 1 Batch 104\n",
      "Epoch 1 Batch 105\n",
      "Epoch 1 Batch 106\n",
      "Epoch 1 Batch 107\n",
      "Epoch 1 Batch 108\n",
      "Epoch 1 Batch 109\n",
      "Epoch 1 Batch 110\n",
      "Epoch 1 Batch 111\n",
      "Epoch 1 Batch 112\n",
      "Epoch 1 Batch 113\n",
      "Epoch 1 Batch 114\n",
      "Epoch 1 Batch 115\n",
      "Epoch 1 Batch 116\n",
      "Epoch 1 Batch 117\n",
      "Epoch 1 Batch 118\n",
      "Epoch 1 Batch 119\n",
      "Epoch 1 Batch 120\n",
      "Epoch 1 Batch 121\n",
      "Epoch 1 Batch 122\n",
      "Epoch 1 Batch 123\n",
      "Epoch 1 Batch 124\n",
      "Epoch 1 Batch 125\n",
      "Epoch 1 Batch 126\n",
      "Epoch 1 Batch 127\n",
      "Epoch 1 Batch 128\n",
      "Epoch 1 Batch 129\n",
      "Epoch 1 Batch 130\n",
      "Epoch 1 Batch 131\n",
      "Epoch 1 Batch 132\n",
      "Epoch 1 Batch 133\n",
      "Epoch 1 Batch 134\n",
      "Epoch 1 Batch 135\n",
      "Epoch 1 Batch 136\n",
      "Epoch 1 Batch 137\n",
      "Epoch 1 Batch 138\n",
      "Epoch 1 Batch 139\n",
      "Epoch 1 Batch 140\n",
      "Epoch 1 Batch 141\n",
      "Epoch 1 Batch 142\n",
      "Epoch 1 Batch 143\n",
      "Epoch 1 Batch 144\n",
      "Epoch 1 Batch 145\n",
      "Epoch 1 Batch 146\n",
      "Epoch 1 Batch 147\n",
      "Epoch 1 Batch 148\n",
      "Epoch 1 Batch 149\n",
      "Epoch 1 Batch 150\n",
      "Epoch 1 Batch 151\n",
      "Epoch 1 Batch 152\n",
      "Epoch 1 Batch 153\n",
      "Epoch 1 Batch 154\n",
      "Epoch 1 Batch 155\n",
      "Epoch 1 Batch 156\n",
      "Epoch 1 Batch 157\n",
      "Epoch 1 Batch 158\n",
      "Epoch 1 Batch 159\n",
      "Epoch 1 Batch 160\n",
      "Epoch 1 Batch 161\n",
      "Epoch 1 Batch 162\n",
      "Epoch 1 Batch 163\n",
      "Epoch 1 Batch 164\n",
      "Epoch 1 Batch 165\n",
      "Epoch 1 Batch 166\n",
      "Epoch 1 Batch 167\n",
      "Epoch 1 Batch 168\n",
      "Epoch 1 Batch 169\n",
      "Epoch 1 Batch 170\n",
      "Epoch 1 Batch 171\n",
      "Epoch 1 Batch 172\n",
      "Epoch 1 Batch 173\n",
      "Epoch 1 Batch 174\n",
      "Epoch 1 Batch 175\n",
      "Epoch 1 Batch 176\n",
      "Epoch 1 Batch 177\n",
      "Epoch 1 Batch 178\n",
      "Epoch 1 Batch 179\n",
      "Epoch 1 Batch 180\n",
      "Epoch 1 Batch 181\n",
      "Epoch 1 Batch 182\n",
      "Epoch 1 Batch 183\n",
      "Epoch 1 Batch 184\n",
      "Epoch 1 Batch 185\n",
      "Epoch 1 Batch 186\n",
      "Epoch 1 Batch 187\n",
      "Epoch 1 Batch 188\n",
      "Epoch 1 Batch 189\n",
      "Epoch 1 Batch 190\n",
      "Epoch 1 Batch 191\n",
      "Epoch 1 Batch 192\n",
      "Epoch 1 Batch 193\n",
      "Epoch 1 Batch 194\n",
      "Epoch 1 Batch 195\n",
      "Epoch 1 Batch 196\n",
      "Epoch 1 Batch 197\n",
      "Epoch 1 Batch 198\n",
      "Epoch 1 Batch 199\n",
      "Epoch 1 Batch 200\n",
      "Epoch 1 Batch 201\n",
      "Epoch 1 Batch 202\n",
      "Epoch 1 Batch 203\n",
      "Epoch 1 Batch 204\n",
      "Epoch 1 Batch 205\n",
      "Epoch 1 Batch 206\n",
      "Epoch 1 Batch 207\n",
      "Epoch 1 Batch 208\n",
      "Epoch 1 Batch 209\n",
      "Epoch 1 Batch 210\n",
      "Epoch 1 Batch 211\n",
      "Epoch 1 Batch 212\n",
      "Epoch 1 Batch 213\n",
      "Epoch 1 Batch 214\n",
      "Epoch 1 Batch 215\n",
      "Epoch 1 Batch 216\n",
      "Epoch 1 Batch 217\n",
      "Epoch 1 Batch 218\n",
      "Epoch 1 Batch 219\n",
      "Epoch 1 Batch 220\n",
      "Epoch 1 Batch 221\n",
      "Epoch 1 Batch 222\n",
      "Epoch 1 Batch 223\n",
      "Epoch 1 Batch 224\n",
      "Epoch 1 Batch 225\n",
      "Epoch 1 Batch 226\n",
      "Epoch 1 Batch 227\n",
      "Epoch 1 Batch 228\n",
      "Epoch 1 Batch 229\n",
      "Epoch 1 Batch 230\n",
      "Epoch 1 Batch 231\n",
      "Epoch 1 Batch 232\n",
      "Epoch 1 Batch 233\n",
      "Epoch 1 Batch 234\n",
      "Epoch 1 Batch 235\n",
      "Epoch 1 Batch 236\n",
      "Epoch 1 Batch 237\n",
      "Epoch 1 Batch 238\n",
      "Epoch 1 Batch 239\n",
      "Epoch 1 Batch 240\n",
      "Epoch 1 Batch 241\n",
      "Epoch 1 Batch 242\n",
      "Epoch 1 Batch 243\n",
      "Epoch 1 Batch 244\n",
      "Epoch 1 Batch 245\n",
      "Epoch 1 Batch 246\n",
      "Epoch 1 Batch 247\n",
      "Epoch 1 Batch 248\n",
      "Epoch 1 Batch 249\n",
      "Epoch 1 Batch 250\n",
      "Epoch 1 Batch 251\n",
      "Epoch 1 Batch 252\n",
      "Epoch 1 Batch 253\n",
      "Epoch 1 Batch 254\n",
      "Epoch 1 Batch 255\n",
      "Epoch 1 Batch 256\n",
      "Epoch 1 Batch 257\n",
      "Epoch 1 Batch 258\n",
      "Epoch 1 Batch 259\n",
      "Epoch 1 Batch 260\n",
      "Epoch 1 Batch 261\n",
      "Epoch 1 Batch 262\n",
      "Epoch 1 Batch 263\n",
      "Epoch 1 Batch 264\n",
      "Epoch 1 Batch 265\n",
      "Epoch 1 Batch 266\n",
      "Epoch 1 Batch 267\n",
      "Epoch 1 Batch 268\n",
      "Epoch 1 Batch 269\n",
      "Epoch 1 Batch 270\n",
      "Epoch 1 Batch 271\n",
      "Epoch 1 Batch 272\n",
      "Epoch 1 Batch 273\n",
      "Epoch 1 Batch 274\n",
      "Epoch 1 Batch 275\n",
      "Epoch 1 Batch 276\n",
      "Epoch 1 Batch 277\n",
      "Epoch 1 Batch 278\n",
      "Epoch 1 Batch 279\n",
      "Epoch 1 Batch 280\n",
      "Epoch 1 Batch 281\n",
      "Epoch 1 Batch 282\n",
      "Epoch 1 Batch 283\n",
      "Epoch 1 Batch 284\n",
      "Epoch 1 Batch 285\n",
      "Epoch 1 Batch 286\n",
      "Epoch 1 Batch 287\n",
      "Epoch 1 Batch 288\n",
      "Epoch 1 Batch 289\n",
      "Epoch 1 Batch 290\n",
      "Epoch 1 Batch 291\n",
      "Epoch 1 Batch 292\n",
      "Epoch 1 Batch 293\n",
      "Epoch 1 Batch 294\n",
      "Epoch 1 Batch 295\n",
      "Epoch 1 Batch 296\n",
      "Epoch 1 Batch 297\n",
      "Epoch 1 Batch 298\n",
      "Epoch 1 Batch 299\n",
      "Epoch 1 Batch 300\n",
      "Epoch 1 Batch 301\n",
      "Epoch 1 Batch 302\n",
      "Epoch 1 Batch 303\n",
      "Epoch 1 Batch 304\n",
      "Epoch 1 Batch 305\n",
      "Epoch 1 Batch 306\n",
      "Epoch 1 Batch 307\n",
      "Epoch 1 Batch 308\n",
      "Epoch 1 Batch 309\n",
      "Epoch 1 Batch 310\n",
      "Epoch 1 Batch 311\n",
      "Epoch 1 Batch 312\n",
      "Epoch 1 Batch 313\n",
      "Epoch 1 Batch 314\n",
      "Epoch 1 Batch 315\n",
      "Epoch 1 Batch 316\n",
      "Epoch 1 Batch 317\n",
      "Epoch 1 Batch 318\n",
      "Epoch 1 Batch 319\n",
      "Epoch 1 Batch 320\n",
      "Epoch 1 Batch 321\n",
      "Epoch 1 Batch 322\n",
      "Epoch 1 Batch 323\n",
      "Epoch 1 Batch 324\n",
      "Epoch 1 Batch 325\n",
      "Epoch 1 Batch 326\n",
      "Epoch 1 Batch 327\n",
      "Epoch 1 Batch 328\n",
      "Epoch 1 Batch 329\n",
      "Epoch 1 Batch 330\n",
      "Epoch 1 Batch 331\n",
      "Epoch 1 Batch 332\n",
      "Epoch 1 Batch 333\n",
      "Epoch 1 Batch 334\n",
      "Epoch 1 Batch 335\n",
      "Epoch 1 Batch 336\n",
      "Epoch 1 Batch 337\n",
      "Epoch 1 Batch 338\n",
      "Epoch 1 Batch 339\n",
      "Epoch 1 Batch 340\n",
      "Epoch 1 Batch 341\n",
      "Epoch 1 Batch 342\n",
      "Epoch 1 Batch 343\n",
      "Epoch 1 Batch 344\n",
      "Epoch 1 Batch 345\n",
      "Epoch 1 Batch 346\n",
      "Epoch 1 Batch 347\n",
      "Epoch 1 Batch 348\n",
      "Epoch 1 Batch 349\n",
      "Epoch 1 Batch 350\n",
      "Epoch 1 Batch 351\n",
      "Epoch 1 Batch 352\n",
      "Epoch 1 Batch 353\n",
      "Epoch 1 Batch 354\n",
      "Epoch 1 Batch 355\n",
      "Epoch 1 Batch 356\n",
      "Epoch 1 Batch 357\n",
      "Epoch 1 Batch 358\n",
      "Epoch 1 Batch 359\n",
      "Epoch 1 Batch 360\n",
      "Epoch 1 Batch 361\n",
      "Epoch 1 Batch 362\n",
      "Epoch 1 Batch 363\n",
      "Epoch 1 Batch 364\n",
      "Epoch 1 Batch 365\n",
      "Epoch 1 Batch 366\n",
      "Epoch 1 Batch 367\n",
      "Epoch 1 Batch 368\n",
      "Epoch 1 Batch 369\n",
      "Epoch 1 Batch 370\n",
      "Epoch 1 Batch 371\n",
      "Epoch 1 Batch 372\n",
      "Epoch 1 Batch 373\n",
      "Epoch 1 Batch 374\n",
      "Epoch 1 Batch 375\n",
      "Epoch 1 Batch 376\n",
      "Epoch 1 Batch 377\n",
      "Epoch 1 Batch 378\n",
      "Epoch 1 Batch 379\n",
      "Epoch 1 Batch 380\n",
      "Epoch 1 Batch 381\n",
      "Epoch 1 Batch 382\n",
      "Epoch 1 Batch 383\n",
      "Epoch 1 Batch 384\n",
      "Epoch 1 Batch 385\n",
      "Epoch 1 Batch 386\n",
      "Epoch 1 Batch 387\n",
      "Epoch 1 Batch 388\n",
      "Epoch 1 Batch 389\n",
      "Epoch 1 Batch 390\n",
      "Epoch 2 Batch 1\n",
      "Epoch 2 Batch 2\n",
      "Epoch 2 Batch 3\n",
      "Epoch 2 Batch 4\n",
      "Epoch 2 Batch 5\n",
      "Epoch 2 Batch 6\n",
      "Epoch 2 Batch 7\n",
      "Epoch 2 Batch 8\n",
      "Epoch 2 Batch 9\n",
      "Epoch 2 Batch 10\n",
      "Epoch 2 Batch 11\n",
      "Epoch 2 Batch 12\n",
      "Epoch 2 Batch 13\n",
      "Epoch 2 Batch 14\n",
      "Epoch 2 Batch 15\n",
      "Epoch 2 Batch 16\n",
      "Epoch 2 Batch 17\n",
      "Epoch 2 Batch 18\n",
      "Epoch 2 Batch 19\n",
      "Epoch 2 Batch 20\n",
      "Epoch 2 Batch 21\n",
      "Epoch 2 Batch 22\n",
      "Epoch 2 Batch 23\n",
      "Epoch 2 Batch 24\n",
      "Epoch 2 Batch 25\n",
      "Epoch 2 Batch 26\n",
      "Epoch 2 Batch 27\n",
      "Epoch 2 Batch 28\n",
      "Epoch 2 Batch 29\n",
      "Epoch 2 Batch 30\n",
      "Epoch 2 Batch 31\n",
      "Epoch 2 Batch 32\n",
      "Epoch 2 Batch 33\n",
      "Epoch 2 Batch 34\n",
      "Epoch 2 Batch 35\n",
      "Epoch 2 Batch 36\n",
      "Epoch 2 Batch 37\n",
      "Epoch 2 Batch 38\n",
      "Epoch 2 Batch 39\n",
      "Epoch 2 Batch 40\n",
      "Epoch 2 Batch 41\n",
      "Epoch 2 Batch 42\n",
      "Epoch 2 Batch 43\n",
      "Epoch 2 Batch 44\n",
      "Epoch 2 Batch 45\n",
      "Epoch 2 Batch 46\n",
      "Epoch 2 Batch 47\n",
      "Epoch 2 Batch 48\n",
      "Epoch 2 Batch 49\n",
      "Epoch 2 Batch 50\n",
      "Epoch 2 Batch 51\n",
      "Epoch 2 Batch 52\n",
      "Epoch 2 Batch 53\n",
      "Epoch 2 Batch 54\n",
      "Epoch 2 Batch 55\n",
      "Epoch 2 Batch 56\n",
      "Epoch 2 Batch 57\n",
      "Epoch 2 Batch 58\n",
      "Epoch 2 Batch 59\n",
      "Epoch 2 Batch 60\n",
      "Epoch 2 Batch 61\n",
      "Epoch 2 Batch 62\n",
      "Epoch 2 Batch 63\n",
      "Epoch 2 Batch 64\n",
      "Epoch 2 Batch 65\n",
      "Epoch 2 Batch 66\n",
      "Epoch 2 Batch 67\n",
      "Epoch 2 Batch 68\n",
      "Epoch 2 Batch 69\n",
      "Epoch 2 Batch 70\n",
      "Epoch 2 Batch 71\n",
      "Epoch 2 Batch 72\n",
      "Epoch 2 Batch 73\n",
      "Epoch 2 Batch 74\n",
      "Epoch 2 Batch 75\n",
      "Epoch 2 Batch 76\n",
      "Epoch 2 Batch 77\n",
      "Epoch 2 Batch 78\n",
      "Epoch 2 Batch 79\n",
      "Epoch 2 Batch 80\n",
      "Epoch 2 Batch 81\n",
      "Epoch 2 Batch 82\n",
      "Epoch 2 Batch 83\n",
      "Epoch 2 Batch 84\n",
      "Epoch 2 Batch 85\n",
      "Epoch 2 Batch 86\n",
      "Epoch 2 Batch 87\n",
      "Epoch 2 Batch 88\n",
      "Epoch 2 Batch 89\n",
      "Epoch 2 Batch 90\n",
      "Epoch 2 Batch 91\n",
      "Epoch 2 Batch 92\n",
      "Epoch 2 Batch 93\n",
      "Epoch 2 Batch 94\n",
      "Epoch 2 Batch 95\n",
      "Epoch 2 Batch 96\n",
      "Epoch 2 Batch 97\n",
      "Epoch 2 Batch 98\n",
      "Epoch 2 Batch 99\n",
      "Epoch 2 Batch 100\n",
      "Epoch 2 Batch 101\n",
      "Epoch 2 Batch 102\n",
      "Epoch 2 Batch 103\n",
      "Epoch 2 Batch 104\n",
      "Epoch 2 Batch 105\n",
      "Epoch 2 Batch 106\n",
      "Epoch 2 Batch 107\n",
      "Epoch 2 Batch 108\n",
      "Epoch 2 Batch 109\n",
      "Epoch 2 Batch 110\n",
      "Epoch 2 Batch 111\n",
      "Epoch 2 Batch 112\n",
      "Epoch 2 Batch 113\n",
      "Epoch 2 Batch 114\n",
      "Epoch 2 Batch 115\n",
      "Epoch 2 Batch 116\n",
      "Epoch 2 Batch 117\n",
      "Epoch 2 Batch 118\n",
      "Epoch 2 Batch 119\n",
      "Epoch 2 Batch 120\n",
      "Epoch 2 Batch 121\n",
      "Epoch 2 Batch 122\n",
      "Epoch 2 Batch 123\n",
      "Epoch 2 Batch 124\n",
      "Epoch 2 Batch 125\n",
      "Epoch 2 Batch 126\n",
      "Epoch 2 Batch 127\n",
      "Epoch 2 Batch 128\n",
      "Epoch 2 Batch 129\n",
      "Epoch 2 Batch 130\n",
      "Epoch 2 Batch 131\n",
      "Epoch 2 Batch 132\n",
      "Epoch 2 Batch 133\n",
      "Epoch 2 Batch 134\n",
      "Epoch 2 Batch 135\n",
      "Epoch 2 Batch 136\n",
      "Epoch 2 Batch 137\n",
      "Epoch 2 Batch 138\n",
      "Epoch 2 Batch 139\n",
      "Epoch 2 Batch 140\n",
      "Epoch 2 Batch 141\n",
      "Epoch 2 Batch 142\n",
      "Epoch 2 Batch 143\n",
      "Epoch 2 Batch 144\n",
      "Epoch 2 Batch 145\n",
      "Epoch 2 Batch 146\n",
      "Epoch 2 Batch 147\n",
      "Epoch 2 Batch 148\n",
      "Epoch 2 Batch 149\n",
      "Epoch 2 Batch 150\n",
      "Epoch 2 Batch 151\n",
      "Epoch 2 Batch 152\n",
      "Epoch 2 Batch 153\n",
      "Epoch 2 Batch 154\n",
      "Epoch 2 Batch 155\n",
      "Epoch 2 Batch 156\n",
      "Epoch 2 Batch 157\n",
      "Epoch 2 Batch 158\n",
      "Epoch 2 Batch 159\n",
      "Epoch 2 Batch 160\n",
      "Epoch 2 Batch 161\n",
      "Epoch 2 Batch 162\n",
      "Epoch 2 Batch 163\n",
      "Epoch 2 Batch 164\n",
      "Epoch 2 Batch 165\n",
      "Epoch 2 Batch 166\n",
      "Epoch 2 Batch 167\n",
      "Epoch 2 Batch 168\n",
      "Epoch 2 Batch 169\n",
      "Epoch 2 Batch 170\n",
      "Epoch 2 Batch 171\n",
      "Epoch 2 Batch 172\n",
      "Epoch 2 Batch 173\n",
      "Epoch 2 Batch 174\n",
      "Epoch 2 Batch 175\n",
      "Epoch 2 Batch 176\n",
      "Epoch 2 Batch 177\n",
      "Epoch 2 Batch 178\n",
      "Epoch 2 Batch 179\n",
      "Epoch 2 Batch 180\n",
      "Epoch 2 Batch 181\n",
      "Epoch 2 Batch 182\n",
      "Epoch 2 Batch 183\n",
      "Epoch 2 Batch 184\n",
      "Epoch 2 Batch 185\n",
      "Epoch 2 Batch 186\n",
      "Epoch 2 Batch 187\n",
      "Epoch 2 Batch 188\n",
      "Epoch 2 Batch 189\n",
      "Epoch 2 Batch 190\n",
      "Epoch 2 Batch 191\n",
      "Epoch 2 Batch 192\n",
      "Epoch 2 Batch 193\n",
      "Epoch 2 Batch 194\n",
      "Epoch 2 Batch 195\n",
      "Epoch 2 Batch 196\n",
      "Epoch 2 Batch 197\n",
      "Epoch 2 Batch 198\n",
      "Epoch 2 Batch 199\n",
      "Epoch 2 Batch 200\n",
      "Epoch 2 Batch 201\n",
      "Epoch 2 Batch 202\n",
      "Epoch 2 Batch 203\n",
      "Epoch 2 Batch 204\n",
      "Epoch 2 Batch 205\n",
      "Epoch 2 Batch 206\n",
      "Epoch 2 Batch 207\n",
      "Epoch 2 Batch 208\n",
      "Epoch 2 Batch 209\n",
      "Epoch 2 Batch 210\n",
      "Epoch 2 Batch 211\n",
      "Epoch 2 Batch 212\n",
      "Epoch 2 Batch 213\n",
      "Epoch 2 Batch 214\n",
      "Epoch 2 Batch 215\n",
      "Epoch 2 Batch 216\n",
      "Epoch 2 Batch 217\n",
      "Epoch 2 Batch 218\n",
      "Epoch 2 Batch 219\n",
      "Epoch 2 Batch 220\n",
      "Epoch 2 Batch 221\n",
      "Epoch 2 Batch 222\n",
      "Epoch 2 Batch 223\n",
      "Epoch 2 Batch 224\n",
      "Epoch 2 Batch 225\n",
      "Epoch 2 Batch 226\n",
      "Epoch 2 Batch 227\n",
      "Epoch 2 Batch 228\n",
      "Epoch 2 Batch 229\n",
      "Epoch 2 Batch 230\n",
      "Epoch 2 Batch 231\n",
      "Epoch 2 Batch 232\n",
      "Epoch 2 Batch 233\n",
      "Epoch 2 Batch 234\n",
      "Epoch 2 Batch 235\n",
      "Epoch 2 Batch 236\n",
      "Epoch 2 Batch 237\n",
      "Epoch 2 Batch 238\n",
      "Epoch 2 Batch 239\n",
      "Epoch 2 Batch 240\n",
      "Epoch 2 Batch 241\n",
      "Epoch 2 Batch 242\n",
      "Epoch 2 Batch 243\n",
      "Epoch 2 Batch 244\n",
      "Epoch 2 Batch 245\n",
      "Epoch 2 Batch 246\n",
      "Epoch 2 Batch 247\n",
      "Epoch 2 Batch 248\n",
      "Epoch 2 Batch 249\n",
      "Epoch 2 Batch 250\n",
      "Epoch 2 Batch 251\n",
      "Epoch 2 Batch 252\n",
      "Epoch 2 Batch 253\n",
      "Epoch 2 Batch 254\n",
      "Epoch 2 Batch 255\n",
      "Epoch 2 Batch 256\n",
      "Epoch 2 Batch 257\n",
      "Epoch 2 Batch 258\n",
      "Epoch 2 Batch 259\n",
      "Epoch 2 Batch 260\n",
      "Epoch 2 Batch 261\n",
      "Epoch 2 Batch 262\n",
      "Epoch 2 Batch 263\n",
      "Epoch 2 Batch 264\n",
      "Epoch 2 Batch 265\n",
      "Epoch 2 Batch 266\n",
      "Epoch 2 Batch 267\n",
      "Epoch 2 Batch 268\n",
      "Epoch 2 Batch 269\n",
      "Epoch 2 Batch 270\n",
      "Epoch 2 Batch 271\n",
      "Epoch 2 Batch 272\n",
      "Epoch 2 Batch 273\n",
      "Epoch 2 Batch 274\n",
      "Epoch 2 Batch 275\n",
      "Epoch 2 Batch 276\n",
      "Epoch 2 Batch 277\n",
      "Epoch 2 Batch 278\n",
      "Epoch 2 Batch 279\n",
      "Epoch 2 Batch 280\n",
      "Epoch 2 Batch 281\n",
      "Epoch 2 Batch 282\n",
      "Epoch 2 Batch 283\n",
      "Epoch 2 Batch 284\n",
      "Epoch 2 Batch 285\n",
      "Epoch 2 Batch 286\n",
      "Epoch 2 Batch 287\n",
      "Epoch 2 Batch 288\n",
      "Epoch 2 Batch 289\n",
      "Epoch 2 Batch 290\n",
      "Epoch 2 Batch 291\n",
      "Epoch 2 Batch 292\n",
      "Epoch 2 Batch 293\n",
      "Epoch 2 Batch 294\n",
      "Epoch 2 Batch 295\n",
      "Epoch 2 Batch 296\n",
      "Epoch 2 Batch 297\n",
      "Epoch 2 Batch 298\n",
      "Epoch 2 Batch 299\n",
      "Epoch 2 Batch 300\n",
      "Epoch 2 Batch 301\n",
      "Epoch 2 Batch 302\n",
      "Epoch 2 Batch 303\n",
      "Epoch 2 Batch 304\n",
      "Epoch 2 Batch 305\n",
      "Epoch 2 Batch 306\n",
      "Epoch 2 Batch 307\n",
      "Epoch 2 Batch 308\n",
      "Epoch 2 Batch 309\n",
      "Epoch 2 Batch 310\n",
      "Epoch 2 Batch 311\n",
      "Epoch 2 Batch 312\n",
      "Epoch 2 Batch 313\n",
      "Epoch 2 Batch 314\n",
      "Epoch 2 Batch 315\n",
      "Epoch 2 Batch 316\n",
      "Epoch 2 Batch 317\n",
      "Epoch 2 Batch 318\n",
      "Epoch 2 Batch 319\n",
      "Epoch 2 Batch 320\n",
      "Epoch 2 Batch 321\n",
      "Epoch 2 Batch 322\n",
      "Epoch 2 Batch 323\n",
      "Epoch 2 Batch 324\n",
      "Epoch 2 Batch 325\n",
      "Epoch 2 Batch 326\n",
      "Epoch 2 Batch 327\n",
      "Epoch 2 Batch 328\n",
      "Epoch 2 Batch 329\n",
      "Epoch 2 Batch 330\n",
      "Epoch 2 Batch 331\n",
      "Epoch 2 Batch 332\n",
      "Epoch 2 Batch 333\n",
      "Epoch 2 Batch 334\n",
      "Epoch 2 Batch 335\n",
      "Epoch 2 Batch 336\n",
      "Epoch 2 Batch 337\n",
      "Epoch 2 Batch 338\n",
      "Epoch 2 Batch 339\n",
      "Epoch 2 Batch 340\n",
      "Epoch 2 Batch 341\n",
      "Epoch 2 Batch 342\n",
      "Epoch 2 Batch 343\n",
      "Epoch 2 Batch 344\n",
      "Epoch 2 Batch 345\n",
      "Epoch 2 Batch 346\n",
      "Epoch 2 Batch 347\n",
      "Epoch 2 Batch 348\n",
      "Epoch 2 Batch 349\n",
      "Epoch 2 Batch 350\n",
      "Epoch 2 Batch 351\n",
      "Epoch 2 Batch 352\n",
      "Epoch 2 Batch 353\n",
      "Epoch 2 Batch 354\n",
      "Epoch 2 Batch 355\n",
      "Epoch 2 Batch 356\n",
      "Epoch 2 Batch 357\n",
      "Epoch 2 Batch 358\n",
      "Epoch 2 Batch 359\n",
      "Epoch 2 Batch 360\n",
      "Epoch 2 Batch 361\n",
      "Epoch 2 Batch 362\n",
      "Epoch 2 Batch 363\n",
      "Epoch 2 Batch 364\n",
      "Epoch 2 Batch 365\n",
      "Epoch 2 Batch 366\n",
      "Epoch 2 Batch 367\n",
      "Epoch 2 Batch 368\n",
      "Epoch 2 Batch 369\n",
      "Epoch 2 Batch 370\n",
      "Epoch 2 Batch 371\n",
      "Epoch 2 Batch 372\n",
      "Epoch 2 Batch 373\n",
      "Epoch 2 Batch 374\n",
      "Epoch 2 Batch 375\n",
      "Epoch 2 Batch 376\n",
      "Epoch 2 Batch 377\n",
      "Epoch 2 Batch 378\n",
      "Epoch 2 Batch 379\n",
      "Epoch 2 Batch 380\n",
      "Epoch 2 Batch 381\n",
      "Epoch 2 Batch 382\n",
      "Epoch 2 Batch 383\n",
      "Epoch 2 Batch 384\n",
      "Epoch 2 Batch 385\n",
      "Epoch 2 Batch 386\n",
      "Epoch 2 Batch 387\n",
      "Epoch 2 Batch 388\n",
      "Epoch 2 Batch 389\n",
      "Epoch 2 Batch 390\n",
      "Epoch 3 Batch 1\n",
      "Epoch 3 Batch 2\n",
      "Epoch 3 Batch 3\n",
      "Epoch 3 Batch 4\n",
      "Epoch 3 Batch 5\n",
      "Epoch 3 Batch 6\n",
      "Epoch 3 Batch 7\n",
      "Epoch 3 Batch 8\n",
      "Epoch 3 Batch 9\n",
      "Epoch 3 Batch 10\n",
      "Epoch 3 Batch 11\n",
      "Epoch 3 Batch 12\n",
      "Epoch 3 Batch 13\n",
      "Epoch 3 Batch 14\n",
      "Epoch 3 Batch 15\n",
      "Epoch 3 Batch 16\n",
      "Epoch 3 Batch 17\n",
      "Epoch 3 Batch 18\n",
      "Epoch 3 Batch 19\n",
      "Epoch 3 Batch 20\n",
      "Epoch 3 Batch 21\n",
      "Epoch 3 Batch 22\n",
      "Epoch 3 Batch 23\n",
      "Epoch 3 Batch 24\n",
      "Epoch 3 Batch 25\n",
      "Epoch 3 Batch 26\n",
      "Epoch 3 Batch 27\n",
      "Epoch 3 Batch 28\n",
      "Epoch 3 Batch 29\n",
      "Epoch 3 Batch 30\n",
      "Epoch 3 Batch 31\n",
      "Epoch 3 Batch 32\n",
      "Epoch 3 Batch 33\n",
      "Epoch 3 Batch 34\n",
      "Epoch 3 Batch 35\n",
      "Epoch 3 Batch 36\n",
      "Epoch 3 Batch 37\n",
      "Epoch 3 Batch 38\n",
      "Epoch 3 Batch 39\n",
      "Epoch 3 Batch 40\n",
      "Epoch 3 Batch 41\n",
      "Epoch 3 Batch 42\n",
      "Epoch 3 Batch 43\n",
      "Epoch 3 Batch 44\n",
      "Epoch 3 Batch 45\n",
      "Epoch 3 Batch 46\n",
      "Epoch 3 Batch 47\n",
      "Epoch 3 Batch 48\n",
      "Epoch 3 Batch 49\n",
      "Epoch 3 Batch 50\n",
      "Epoch 3 Batch 51\n",
      "Epoch 3 Batch 52\n",
      "Epoch 3 Batch 53\n",
      "Epoch 3 Batch 54\n",
      "Epoch 3 Batch 55\n",
      "Epoch 3 Batch 56\n",
      "Epoch 3 Batch 57\n",
      "Epoch 3 Batch 58\n",
      "Epoch 3 Batch 59\n",
      "Epoch 3 Batch 60\n",
      "Epoch 3 Batch 61\n",
      "Epoch 3 Batch 62\n",
      "Epoch 3 Batch 63\n",
      "Epoch 3 Batch 64\n",
      "Epoch 3 Batch 65\n",
      "Epoch 3 Batch 66\n",
      "Epoch 3 Batch 67\n",
      "Epoch 3 Batch 68\n",
      "Epoch 3 Batch 69\n",
      "Epoch 3 Batch 70\n",
      "Epoch 3 Batch 71\n",
      "Epoch 3 Batch 72\n",
      "Epoch 3 Batch 73\n",
      "Epoch 3 Batch 74\n",
      "Epoch 3 Batch 75\n",
      "Epoch 3 Batch 76\n",
      "Epoch 3 Batch 77\n",
      "Epoch 3 Batch 78\n",
      "Epoch 3 Batch 79\n",
      "Epoch 3 Batch 80\n",
      "Epoch 3 Batch 81\n",
      "Epoch 3 Batch 82\n",
      "Epoch 3 Batch 83\n",
      "Epoch 3 Batch 84\n",
      "Epoch 3 Batch 85\n",
      "Epoch 3 Batch 86\n",
      "Epoch 3 Batch 87\n",
      "Epoch 3 Batch 88\n",
      "Epoch 3 Batch 89\n",
      "Epoch 3 Batch 90\n",
      "Epoch 3 Batch 91\n",
      "Epoch 3 Batch 92\n",
      "Epoch 3 Batch 93\n",
      "Epoch 3 Batch 94\n",
      "Epoch 3 Batch 95\n",
      "Epoch 3 Batch 96\n",
      "Epoch 3 Batch 97\n",
      "Epoch 3 Batch 98\n",
      "Epoch 3 Batch 99\n",
      "Epoch 3 Batch 100\n",
      "Epoch 3 Batch 101\n",
      "Epoch 3 Batch 102\n",
      "Epoch 3 Batch 103\n",
      "Epoch 3 Batch 104\n",
      "Epoch 3 Batch 105\n",
      "Epoch 3 Batch 106\n",
      "Epoch 3 Batch 107\n",
      "Epoch 3 Batch 108\n",
      "Epoch 3 Batch 109\n",
      "Epoch 3 Batch 110\n",
      "Epoch 3 Batch 111\n",
      "Epoch 3 Batch 112\n",
      "Epoch 3 Batch 113\n",
      "Epoch 3 Batch 114\n",
      "Epoch 3 Batch 115\n",
      "Epoch 3 Batch 116\n",
      "Epoch 3 Batch 117\n",
      "Epoch 3 Batch 118\n",
      "Epoch 3 Batch 119\n",
      "Epoch 3 Batch 120\n",
      "Epoch 3 Batch 121\n",
      "Epoch 3 Batch 122\n",
      "Epoch 3 Batch 123\n",
      "Epoch 3 Batch 124\n",
      "Epoch 3 Batch 125\n",
      "Epoch 3 Batch 126\n",
      "Epoch 3 Batch 127\n",
      "Epoch 3 Batch 128\n",
      "Epoch 3 Batch 129\n",
      "Epoch 3 Batch 130\n",
      "Epoch 3 Batch 131\n",
      "Epoch 3 Batch 132\n",
      "Epoch 3 Batch 133\n",
      "Epoch 3 Batch 134\n",
      "Epoch 3 Batch 135\n",
      "Epoch 3 Batch 136\n",
      "Epoch 3 Batch 137\n",
      "Epoch 3 Batch 138\n",
      "Epoch 3 Batch 139\n",
      "Epoch 3 Batch 140\n",
      "Epoch 3 Batch 141\n",
      "Epoch 3 Batch 142\n",
      "Epoch 3 Batch 143\n",
      "Epoch 3 Batch 144\n",
      "Epoch 3 Batch 145\n",
      "Epoch 3 Batch 146\n",
      "Epoch 3 Batch 147\n",
      "Epoch 3 Batch 148\n",
      "Epoch 3 Batch 149\n",
      "Epoch 3 Batch 150\n",
      "Epoch 3 Batch 151\n",
      "Epoch 3 Batch 152\n",
      "Epoch 3 Batch 153\n",
      "Epoch 3 Batch 154\n",
      "Epoch 3 Batch 155\n",
      "Epoch 3 Batch 156\n",
      "Epoch 3 Batch 157\n",
      "Epoch 3 Batch 158\n",
      "Epoch 3 Batch 159\n",
      "Epoch 3 Batch 160\n",
      "Epoch 3 Batch 161\n",
      "Epoch 3 Batch 162\n",
      "Epoch 3 Batch 163\n",
      "Epoch 3 Batch 164\n",
      "Epoch 3 Batch 165\n",
      "Epoch 3 Batch 166\n",
      "Epoch 3 Batch 167\n",
      "Epoch 3 Batch 168\n",
      "Epoch 3 Batch 169\n",
      "Epoch 3 Batch 170\n",
      "Epoch 3 Batch 171\n",
      "Epoch 3 Batch 172\n",
      "Epoch 3 Batch 173\n",
      "Epoch 3 Batch 174\n",
      "Epoch 3 Batch 175\n",
      "Epoch 3 Batch 176\n",
      "Epoch 3 Batch 177\n",
      "Epoch 3 Batch 178\n",
      "Epoch 3 Batch 179\n",
      "Epoch 3 Batch 180\n",
      "Epoch 3 Batch 181\n",
      "Epoch 3 Batch 182\n",
      "Epoch 3 Batch 183\n",
      "Epoch 3 Batch 184\n",
      "Epoch 3 Batch 185\n",
      "Epoch 3 Batch 186\n",
      "Epoch 3 Batch 187\n",
      "Epoch 3 Batch 188\n",
      "Epoch 3 Batch 189\n",
      "Epoch 3 Batch 190\n",
      "Epoch 3 Batch 191\n",
      "Epoch 3 Batch 192\n",
      "Epoch 3 Batch 193\n",
      "Epoch 3 Batch 194\n",
      "Epoch 3 Batch 195\n",
      "Epoch 3 Batch 196\n",
      "Epoch 3 Batch 197\n",
      "Epoch 3 Batch 198\n",
      "Epoch 3 Batch 199\n",
      "Epoch 3 Batch 200\n",
      "Epoch 3 Batch 201\n",
      "Epoch 3 Batch 202\n",
      "Epoch 3 Batch 203\n",
      "Epoch 3 Batch 204\n",
      "Epoch 3 Batch 205\n",
      "Epoch 3 Batch 206\n",
      "Epoch 3 Batch 207\n",
      "Epoch 3 Batch 208\n",
      "Epoch 3 Batch 209\n",
      "Epoch 3 Batch 210\n",
      "Epoch 3 Batch 211\n",
      "Epoch 3 Batch 212\n",
      "Epoch 3 Batch 213\n",
      "Epoch 3 Batch 214\n",
      "Epoch 3 Batch 215\n",
      "Epoch 3 Batch 216\n",
      "Epoch 3 Batch 217\n",
      "Epoch 3 Batch 218\n",
      "Epoch 3 Batch 219\n",
      "Epoch 3 Batch 220\n",
      "Epoch 3 Batch 221\n",
      "Epoch 3 Batch 222\n",
      "Epoch 3 Batch 223\n",
      "Epoch 3 Batch 224\n",
      "Epoch 3 Batch 225\n",
      "Epoch 3 Batch 226\n",
      "Epoch 3 Batch 227\n",
      "Epoch 3 Batch 228\n",
      "Epoch 3 Batch 229\n",
      "Epoch 3 Batch 230\n",
      "Epoch 3 Batch 231\n",
      "Epoch 3 Batch 232\n",
      "Epoch 3 Batch 233\n",
      "Epoch 3 Batch 234\n",
      "Epoch 3 Batch 235\n",
      "Epoch 3 Batch 236\n",
      "Epoch 3 Batch 237\n",
      "Epoch 3 Batch 238\n",
      "Epoch 3 Batch 239\n",
      "Epoch 3 Batch 240\n",
      "Epoch 3 Batch 241\n",
      "Epoch 3 Batch 242\n",
      "Epoch 3 Batch 243\n",
      "Epoch 3 Batch 244\n",
      "Epoch 3 Batch 245\n",
      "Epoch 3 Batch 246\n",
      "Epoch 3 Batch 247\n",
      "Epoch 3 Batch 248\n",
      "Epoch 3 Batch 249\n",
      "Epoch 3 Batch 250\n",
      "Epoch 3 Batch 251\n",
      "Epoch 3 Batch 252\n",
      "Epoch 3 Batch 253\n",
      "Epoch 3 Batch 254\n",
      "Epoch 3 Batch 255\n",
      "Epoch 3 Batch 256\n",
      "Epoch 3 Batch 257\n",
      "Epoch 3 Batch 258\n",
      "Epoch 3 Batch 259\n",
      "Epoch 3 Batch 260\n",
      "Epoch 3 Batch 261\n",
      "Epoch 3 Batch 262\n",
      "Epoch 3 Batch 263\n",
      "Epoch 3 Batch 264\n",
      "Epoch 3 Batch 265\n",
      "Epoch 3 Batch 266\n",
      "Epoch 3 Batch 267\n",
      "Epoch 3 Batch 268\n",
      "Epoch 3 Batch 269\n",
      "Epoch 3 Batch 270\n",
      "Epoch 3 Batch 271\n",
      "Epoch 3 Batch 272\n",
      "Epoch 3 Batch 273\n",
      "Epoch 3 Batch 274\n",
      "Epoch 3 Batch 275\n",
      "Epoch 3 Batch 276\n",
      "Epoch 3 Batch 277\n",
      "Epoch 3 Batch 278\n",
      "Epoch 3 Batch 279\n",
      "Epoch 3 Batch 280\n",
      "Epoch 3 Batch 281\n",
      "Epoch 3 Batch 282\n",
      "Epoch 3 Batch 283\n",
      "Epoch 3 Batch 284\n",
      "Epoch 3 Batch 285\n",
      "Epoch 3 Batch 286\n",
      "Epoch 3 Batch 287\n",
      "Epoch 3 Batch 288\n",
      "Epoch 3 Batch 289\n",
      "Epoch 3 Batch 290\n",
      "Epoch 3 Batch 291\n",
      "Epoch 3 Batch 292\n",
      "Epoch 3 Batch 293\n",
      "Epoch 3 Batch 294\n",
      "Epoch 3 Batch 295\n",
      "Epoch 3 Batch 296\n",
      "Epoch 3 Batch 297\n",
      "Epoch 3 Batch 298\n",
      "Epoch 3 Batch 299\n",
      "Epoch 3 Batch 300\n",
      "Epoch 3 Batch 301\n",
      "Epoch 3 Batch 302\n",
      "Epoch 3 Batch 303\n",
      "Epoch 3 Batch 304\n",
      "Epoch 3 Batch 305\n",
      "Epoch 3 Batch 306\n",
      "Epoch 3 Batch 307\n",
      "Epoch 3 Batch 308\n",
      "Epoch 3 Batch 309\n",
      "Epoch 3 Batch 310\n",
      "Epoch 3 Batch 311\n",
      "Epoch 3 Batch 312\n",
      "Epoch 3 Batch 313\n",
      "Epoch 3 Batch 314\n",
      "Epoch 3 Batch 315\n",
      "Epoch 3 Batch 316\n",
      "Epoch 3 Batch 317\n",
      "Epoch 3 Batch 318\n",
      "Epoch 3 Batch 319\n",
      "Epoch 3 Batch 320\n",
      "Epoch 3 Batch 321\n",
      "Epoch 3 Batch 322\n",
      "Epoch 3 Batch 323\n",
      "Epoch 3 Batch 324\n",
      "Epoch 3 Batch 325\n",
      "Epoch 3 Batch 326\n",
      "Epoch 3 Batch 327\n",
      "Epoch 3 Batch 328\n",
      "Epoch 3 Batch 329\n",
      "Epoch 3 Batch 330\n",
      "Epoch 3 Batch 331\n",
      "Epoch 3 Batch 332\n",
      "Epoch 3 Batch 333\n",
      "Epoch 3 Batch 334\n",
      "Epoch 3 Batch 335\n",
      "Epoch 3 Batch 336\n",
      "Epoch 3 Batch 337\n",
      "Epoch 3 Batch 338\n",
      "Epoch 3 Batch 339\n",
      "Epoch 3 Batch 340\n",
      "Epoch 3 Batch 341\n",
      "Epoch 3 Batch 342\n",
      "Epoch 3 Batch 343\n",
      "Epoch 3 Batch 344\n",
      "Epoch 3 Batch 345\n",
      "Epoch 3 Batch 346\n",
      "Epoch 3 Batch 347\n",
      "Epoch 3 Batch 348\n",
      "Epoch 3 Batch 349\n",
      "Epoch 3 Batch 350\n",
      "Epoch 3 Batch 351\n",
      "Epoch 3 Batch 352\n",
      "Epoch 3 Batch 353\n",
      "Epoch 3 Batch 354\n",
      "Epoch 3 Batch 355\n",
      "Epoch 3 Batch 356\n",
      "Epoch 3 Batch 357\n",
      "Epoch 3 Batch 358\n",
      "Epoch 3 Batch 359\n",
      "Epoch 3 Batch 360\n",
      "Epoch 3 Batch 361\n",
      "Epoch 3 Batch 362\n",
      "Epoch 3 Batch 363\n",
      "Epoch 3 Batch 364\n",
      "Epoch 3 Batch 365\n",
      "Epoch 3 Batch 366\n",
      "Epoch 3 Batch 367\n",
      "Epoch 3 Batch 368\n",
      "Epoch 3 Batch 369\n",
      "Epoch 3 Batch 370\n",
      "Epoch 3 Batch 371\n",
      "Epoch 3 Batch 372\n",
      "Epoch 3 Batch 373\n",
      "Epoch 3 Batch 374\n",
      "Epoch 3 Batch 375\n",
      "Epoch 3 Batch 376\n",
      "Epoch 3 Batch 377\n",
      "Epoch 3 Batch 378\n",
      "Epoch 3 Batch 379\n",
      "Epoch 3 Batch 380\n",
      "Epoch 3 Batch 381\n",
      "Epoch 3 Batch 382\n",
      "Epoch 3 Batch 383\n",
      "Epoch 3 Batch 384\n",
      "Epoch 3 Batch 385\n",
      "Epoch 3 Batch 386\n",
      "Epoch 3 Batch 387\n",
      "Epoch 3 Batch 388\n",
      "Epoch 3 Batch 389\n",
      "Epoch 3 Batch 390\n",
      "Epoch 4 Batch 1\n",
      "Epoch 4 Batch 2\n",
      "Epoch 4 Batch 3\n",
      "Epoch 4 Batch 4\n",
      "Epoch 4 Batch 5\n",
      "Epoch 4 Batch 6\n",
      "Epoch 4 Batch 7\n",
      "Epoch 4 Batch 8\n",
      "Epoch 4 Batch 9\n",
      "Epoch 4 Batch 10\n",
      "Epoch 4 Batch 11\n",
      "Epoch 4 Batch 12\n",
      "Epoch 4 Batch 13\n",
      "Epoch 4 Batch 14\n",
      "Epoch 4 Batch 15\n",
      "Epoch 4 Batch 16\n",
      "Epoch 4 Batch 17\n",
      "Epoch 4 Batch 18\n",
      "Epoch 4 Batch 19\n",
      "Epoch 4 Batch 20\n",
      "Epoch 4 Batch 21\n",
      "Epoch 4 Batch 22\n",
      "Epoch 4 Batch 23\n",
      "Epoch 4 Batch 24\n",
      "Epoch 4 Batch 25\n",
      "Epoch 4 Batch 26\n",
      "Epoch 4 Batch 27\n",
      "Epoch 4 Batch 28\n",
      "Epoch 4 Batch 29\n",
      "Epoch 4 Batch 30\n",
      "Epoch 4 Batch 31\n",
      "Epoch 4 Batch 32\n",
      "Epoch 4 Batch 33\n",
      "Epoch 4 Batch 34\n",
      "Epoch 4 Batch 35\n",
      "Epoch 4 Batch 36\n",
      "Epoch 4 Batch 37\n",
      "Epoch 4 Batch 38\n",
      "Epoch 4 Batch 39\n",
      "Epoch 4 Batch 40\n",
      "Epoch 4 Batch 41\n",
      "Epoch 4 Batch 42\n",
      "Epoch 4 Batch 43\n",
      "Epoch 4 Batch 44\n",
      "Epoch 4 Batch 45\n",
      "Epoch 4 Batch 46\n",
      "Epoch 4 Batch 47\n",
      "Epoch 4 Batch 48\n",
      "Epoch 4 Batch 49\n",
      "Epoch 4 Batch 50\n",
      "Epoch 4 Batch 51\n",
      "Epoch 4 Batch 52\n",
      "Epoch 4 Batch 53\n",
      "Epoch 4 Batch 54\n",
      "Epoch 4 Batch 55\n",
      "Epoch 4 Batch 56\n",
      "Epoch 4 Batch 57\n",
      "Epoch 4 Batch 58\n",
      "Epoch 4 Batch 59\n",
      "Epoch 4 Batch 60\n",
      "Epoch 4 Batch 61\n",
      "Epoch 4 Batch 62\n",
      "Epoch 4 Batch 63\n",
      "Epoch 4 Batch 64\n",
      "Epoch 4 Batch 65\n",
      "Epoch 4 Batch 66\n",
      "Epoch 4 Batch 67\n",
      "Epoch 4 Batch 68\n",
      "Epoch 4 Batch 69\n",
      "Epoch 4 Batch 70\n",
      "Epoch 4 Batch 71\n",
      "Epoch 4 Batch 72\n",
      "Epoch 4 Batch 73\n",
      "Epoch 4 Batch 74\n",
      "Epoch 4 Batch 75\n",
      "Epoch 4 Batch 76\n",
      "Epoch 4 Batch 77\n",
      "Epoch 4 Batch 78\n",
      "Epoch 4 Batch 79\n",
      "Epoch 4 Batch 80\n",
      "Epoch 4 Batch 81\n",
      "Epoch 4 Batch 82\n",
      "Epoch 4 Batch 83\n",
      "Epoch 4 Batch 84\n",
      "Epoch 4 Batch 85\n",
      "Epoch 4 Batch 86\n",
      "Epoch 4 Batch 87\n",
      "Epoch 4 Batch 88\n",
      "Epoch 4 Batch 89\n",
      "Epoch 4 Batch 90\n",
      "Epoch 4 Batch 91\n",
      "Epoch 4 Batch 92\n",
      "Epoch 4 Batch 93\n",
      "Epoch 4 Batch 94\n",
      "Epoch 4 Batch 95\n",
      "Epoch 4 Batch 96\n",
      "Epoch 4 Batch 97\n",
      "Epoch 4 Batch 98\n",
      "Epoch 4 Batch 99\n",
      "Epoch 4 Batch 100\n",
      "Epoch 4 Batch 101\n",
      "Epoch 4 Batch 102\n",
      "Epoch 4 Batch 103\n",
      "Epoch 4 Batch 104\n",
      "Epoch 4 Batch 105\n",
      "Epoch 4 Batch 106\n",
      "Epoch 4 Batch 107\n",
      "Epoch 4 Batch 108\n",
      "Epoch 4 Batch 109\n",
      "Epoch 4 Batch 110\n",
      "Epoch 4 Batch 111\n",
      "Epoch 4 Batch 112\n",
      "Epoch 4 Batch 113\n",
      "Epoch 4 Batch 114\n",
      "Epoch 4 Batch 115\n",
      "Epoch 4 Batch 116\n",
      "Epoch 4 Batch 117\n",
      "Epoch 4 Batch 118\n",
      "Epoch 4 Batch 119\n",
      "Epoch 4 Batch 120\n",
      "Epoch 4 Batch 121\n",
      "Epoch 4 Batch 122\n",
      "Epoch 4 Batch 123\n",
      "Epoch 4 Batch 124\n",
      "Epoch 4 Batch 125\n",
      "Epoch 4 Batch 126\n",
      "Epoch 4 Batch 127\n",
      "Epoch 4 Batch 128\n",
      "Epoch 4 Batch 129\n",
      "Epoch 4 Batch 130\n",
      "Epoch 4 Batch 131\n",
      "Epoch 4 Batch 132\n",
      "Epoch 4 Batch 133\n",
      "Epoch 4 Batch 134\n",
      "Epoch 4 Batch 135\n",
      "Epoch 4 Batch 136\n",
      "Epoch 4 Batch 137\n",
      "Epoch 4 Batch 138\n",
      "Epoch 4 Batch 139\n",
      "Epoch 4 Batch 140\n",
      "Epoch 4 Batch 141\n",
      "Epoch 4 Batch 142\n",
      "Epoch 4 Batch 143\n",
      "Epoch 4 Batch 144\n",
      "Epoch 4 Batch 145\n",
      "Epoch 4 Batch 146\n",
      "Epoch 4 Batch 147\n",
      "Epoch 4 Batch 148\n",
      "Epoch 4 Batch 149\n",
      "Epoch 4 Batch 150\n",
      "Epoch 4 Batch 151\n",
      "Epoch 4 Batch 152\n",
      "Epoch 4 Batch 153\n",
      "Epoch 4 Batch 154\n",
      "Epoch 4 Batch 155\n",
      "Epoch 4 Batch 156\n",
      "Epoch 4 Batch 157\n",
      "Epoch 4 Batch 158\n",
      "Epoch 4 Batch 159\n",
      "Epoch 4 Batch 160\n",
      "Epoch 4 Batch 161\n",
      "Epoch 4 Batch 162\n",
      "Epoch 4 Batch 163\n",
      "Epoch 4 Batch 164\n",
      "Epoch 4 Batch 165\n",
      "Epoch 4 Batch 166\n",
      "Epoch 4 Batch 167\n",
      "Epoch 4 Batch 168\n",
      "Epoch 4 Batch 169\n",
      "Epoch 4 Batch 170\n",
      "Epoch 4 Batch 171\n",
      "Epoch 4 Batch 172\n",
      "Epoch 4 Batch 173\n",
      "Epoch 4 Batch 174\n",
      "Epoch 4 Batch 175\n",
      "Epoch 4 Batch 176\n",
      "Epoch 4 Batch 177\n",
      "Epoch 4 Batch 178\n",
      "Epoch 4 Batch 179\n",
      "Epoch 4 Batch 180\n",
      "Epoch 4 Batch 181\n",
      "Epoch 4 Batch 182\n",
      "Epoch 4 Batch 183\n",
      "Epoch 4 Batch 184\n",
      "Epoch 4 Batch 185\n",
      "Epoch 4 Batch 186\n",
      "Epoch 4 Batch 187\n",
      "Epoch 4 Batch 188\n",
      "Epoch 4 Batch 189\n",
      "Epoch 4 Batch 190\n",
      "Epoch 4 Batch 191\n",
      "Epoch 4 Batch 192\n",
      "Epoch 4 Batch 193\n",
      "Epoch 4 Batch 194\n",
      "Epoch 4 Batch 195\n",
      "Epoch 4 Batch 196\n",
      "Epoch 4 Batch 197\n",
      "Epoch 4 Batch 198\n",
      "Epoch 4 Batch 199\n",
      "Epoch 4 Batch 200\n",
      "Epoch 4 Batch 201\n",
      "Epoch 4 Batch 202\n",
      "Epoch 4 Batch 203\n",
      "Epoch 4 Batch 204\n",
      "Epoch 4 Batch 205\n",
      "Epoch 4 Batch 206\n",
      "Epoch 4 Batch 207\n",
      "Epoch 4 Batch 208\n",
      "Epoch 4 Batch 209\n",
      "Epoch 4 Batch 210\n",
      "Epoch 4 Batch 211\n",
      "Epoch 4 Batch 212\n",
      "Epoch 4 Batch 213\n",
      "Epoch 4 Batch 214\n",
      "Epoch 4 Batch 215\n",
      "Epoch 4 Batch 216\n",
      "Epoch 4 Batch 217\n",
      "Epoch 4 Batch 218\n",
      "Epoch 4 Batch 219\n",
      "Epoch 4 Batch 220\n",
      "Epoch 4 Batch 221\n",
      "Epoch 4 Batch 222\n",
      "Epoch 4 Batch 223\n",
      "Epoch 4 Batch 224\n",
      "Epoch 4 Batch 225\n",
      "Epoch 4 Batch 226\n",
      "Epoch 4 Batch 227\n",
      "Epoch 4 Batch 228\n",
      "Epoch 4 Batch 229\n",
      "Epoch 4 Batch 230\n",
      "Epoch 4 Batch 231\n",
      "Epoch 4 Batch 232\n",
      "Epoch 4 Batch 233\n",
      "Epoch 4 Batch 234\n",
      "Epoch 4 Batch 235\n",
      "Epoch 4 Batch 236\n",
      "Epoch 4 Batch 237\n",
      "Epoch 4 Batch 238\n",
      "Epoch 4 Batch 239\n",
      "Epoch 4 Batch 240\n",
      "Epoch 4 Batch 241\n",
      "Epoch 4 Batch 242\n",
      "Epoch 4 Batch 243\n",
      "Epoch 4 Batch 244\n",
      "Epoch 4 Batch 245\n",
      "Epoch 4 Batch 246\n",
      "Epoch 4 Batch 247\n",
      "Epoch 4 Batch 248\n",
      "Epoch 4 Batch 249\n",
      "Epoch 4 Batch 250\n",
      "Epoch 4 Batch 251\n",
      "Epoch 4 Batch 252\n",
      "Epoch 4 Batch 253\n",
      "Epoch 4 Batch 254\n",
      "Epoch 4 Batch 255\n",
      "Epoch 4 Batch 256\n",
      "Epoch 4 Batch 257\n",
      "Epoch 4 Batch 258\n",
      "Epoch 4 Batch 259\n",
      "Epoch 4 Batch 260\n",
      "Epoch 4 Batch 261\n",
      "Epoch 4 Batch 262\n",
      "Epoch 4 Batch 263\n",
      "Epoch 4 Batch 264\n",
      "Epoch 4 Batch 265\n",
      "Epoch 4 Batch 266\n",
      "Epoch 4 Batch 267\n",
      "Epoch 4 Batch 268\n",
      "Epoch 4 Batch 269\n",
      "Epoch 4 Batch 270\n",
      "Epoch 4 Batch 271\n",
      "Epoch 4 Batch 272\n",
      "Epoch 4 Batch 273\n",
      "Epoch 4 Batch 274\n",
      "Epoch 4 Batch 275\n",
      "Epoch 4 Batch 276\n",
      "Epoch 4 Batch 277\n",
      "Epoch 4 Batch 278\n",
      "Epoch 4 Batch 279\n",
      "Epoch 4 Batch 280\n",
      "Epoch 4 Batch 281\n",
      "Epoch 4 Batch 282\n",
      "Epoch 4 Batch 283\n",
      "Epoch 4 Batch 284\n",
      "Epoch 4 Batch 285\n",
      "Epoch 4 Batch 286\n",
      "Epoch 4 Batch 287\n",
      "Epoch 4 Batch 288\n",
      "Epoch 4 Batch 289\n",
      "Epoch 4 Batch 290\n",
      "Epoch 4 Batch 291\n",
      "Epoch 4 Batch 292\n",
      "Epoch 4 Batch 293\n",
      "Epoch 4 Batch 294\n",
      "Epoch 4 Batch 295\n",
      "Epoch 4 Batch 296\n",
      "Epoch 4 Batch 297\n",
      "Epoch 4 Batch 298\n",
      "Epoch 4 Batch 299\n",
      "Epoch 4 Batch 300\n",
      "Epoch 4 Batch 301\n",
      "Epoch 4 Batch 302\n",
      "Epoch 4 Batch 303\n",
      "Epoch 4 Batch 304\n",
      "Epoch 4 Batch 305\n",
      "Epoch 4 Batch 306\n",
      "Epoch 4 Batch 307\n",
      "Epoch 4 Batch 308\n",
      "Epoch 4 Batch 309\n",
      "Epoch 4 Batch 310\n",
      "Epoch 4 Batch 311\n",
      "Epoch 4 Batch 312\n",
      "Epoch 4 Batch 313\n",
      "Epoch 4 Batch 314\n",
      "Epoch 4 Batch 315\n",
      "Epoch 4 Batch 316\n",
      "Epoch 4 Batch 317\n",
      "Epoch 4 Batch 318\n",
      "Epoch 4 Batch 319\n",
      "Epoch 4 Batch 320\n",
      "Epoch 4 Batch 321\n",
      "Epoch 4 Batch 322\n",
      "Epoch 4 Batch 323\n",
      "Epoch 4 Batch 324\n",
      "Epoch 4 Batch 325\n",
      "Epoch 4 Batch 326\n",
      "Epoch 4 Batch 327\n",
      "Epoch 4 Batch 328\n",
      "Epoch 4 Batch 329\n",
      "Epoch 4 Batch 330\n",
      "Epoch 4 Batch 331\n",
      "Epoch 4 Batch 332\n",
      "Epoch 4 Batch 333\n",
      "Epoch 4 Batch 334\n",
      "Epoch 4 Batch 335\n",
      "Epoch 4 Batch 336\n",
      "Epoch 4 Batch 337\n",
      "Epoch 4 Batch 338\n",
      "Epoch 4 Batch 339\n",
      "Epoch 4 Batch 340\n",
      "Epoch 4 Batch 341\n",
      "Epoch 4 Batch 342\n",
      "Epoch 4 Batch 343\n",
      "Epoch 4 Batch 344\n",
      "Epoch 4 Batch 345\n",
      "Epoch 4 Batch 346\n",
      "Epoch 4 Batch 347\n",
      "Epoch 4 Batch 348\n",
      "Epoch 4 Batch 349\n",
      "Epoch 4 Batch 350\n",
      "Epoch 4 Batch 351\n",
      "Epoch 4 Batch 352\n",
      "Epoch 4 Batch 353\n",
      "Epoch 4 Batch 354\n",
      "Epoch 4 Batch 355\n",
      "Epoch 4 Batch 356\n",
      "Epoch 4 Batch 357\n",
      "Epoch 4 Batch 358\n",
      "Epoch 4 Batch 359\n",
      "Epoch 4 Batch 360\n",
      "Epoch 4 Batch 361\n",
      "Epoch 4 Batch 362\n",
      "Epoch 4 Batch 363\n",
      "Epoch 4 Batch 364\n",
      "Epoch 4 Batch 365\n",
      "Epoch 4 Batch 366\n",
      "Epoch 4 Batch 367\n",
      "Epoch 4 Batch 368\n",
      "Epoch 4 Batch 369\n",
      "Epoch 4 Batch 370\n",
      "Epoch 4 Batch 371\n",
      "Epoch 4 Batch 372\n",
      "Epoch 4 Batch 373\n",
      "Epoch 4 Batch 374\n",
      "Epoch 4 Batch 375\n",
      "Epoch 4 Batch 376\n",
      "Epoch 4 Batch 377\n",
      "Epoch 4 Batch 378\n",
      "Epoch 4 Batch 379\n",
      "Epoch 4 Batch 380\n",
      "Epoch 4 Batch 381\n",
      "Epoch 4 Batch 382\n",
      "Epoch 4 Batch 383\n",
      "Epoch 4 Batch 384\n",
      "Epoch 4 Batch 385\n",
      "Epoch 4 Batch 386\n",
      "Epoch 4 Batch 387\n",
      "Epoch 4 Batch 388\n",
      "Epoch 4 Batch 389\n",
      "Epoch 4 Batch 390\n",
      "Epoch 5 Batch 1\n",
      "Epoch 5 Batch 2\n",
      "Epoch 5 Batch 3\n",
      "Epoch 5 Batch 4\n",
      "Epoch 5 Batch 5\n",
      "Epoch 5 Batch 6\n",
      "Epoch 5 Batch 7\n",
      "Epoch 5 Batch 8\n",
      "Epoch 5 Batch 9\n",
      "Epoch 5 Batch 10\n",
      "Epoch 5 Batch 11\n",
      "Epoch 5 Batch 12\n",
      "Epoch 5 Batch 13\n",
      "Epoch 5 Batch 14\n",
      "Epoch 5 Batch 15\n",
      "Epoch 5 Batch 16\n",
      "Epoch 5 Batch 17\n",
      "Epoch 5 Batch 18\n",
      "Epoch 5 Batch 19\n",
      "Epoch 5 Batch 20\n",
      "Epoch 5 Batch 21\n",
      "Epoch 5 Batch 22\n",
      "Epoch 5 Batch 23\n",
      "Epoch 5 Batch 24\n",
      "Epoch 5 Batch 25\n",
      "Epoch 5 Batch 26\n",
      "Epoch 5 Batch 27\n",
      "Epoch 5 Batch 28\n",
      "Epoch 5 Batch 29\n",
      "Epoch 5 Batch 30\n",
      "Epoch 5 Batch 31\n",
      "Epoch 5 Batch 32\n",
      "Epoch 5 Batch 33\n",
      "Epoch 5 Batch 34\n",
      "Epoch 5 Batch 35\n",
      "Epoch 5 Batch 36\n",
      "Epoch 5 Batch 37\n",
      "Epoch 5 Batch 38\n",
      "Epoch 5 Batch 39\n",
      "Epoch 5 Batch 40\n",
      "Epoch 5 Batch 41\n",
      "Epoch 5 Batch 42\n",
      "Epoch 5 Batch 43\n",
      "Epoch 5 Batch 44\n",
      "Epoch 5 Batch 45\n",
      "Epoch 5 Batch 46\n",
      "Epoch 5 Batch 47\n",
      "Epoch 5 Batch 48\n",
      "Epoch 5 Batch 49\n",
      "Epoch 5 Batch 50\n",
      "Epoch 5 Batch 51\n",
      "Epoch 5 Batch 52\n",
      "Epoch 5 Batch 53\n",
      "Epoch 5 Batch 54\n",
      "Epoch 5 Batch 55\n",
      "Epoch 5 Batch 56\n",
      "Epoch 5 Batch 57\n",
      "Epoch 5 Batch 58\n",
      "Epoch 5 Batch 59\n",
      "Epoch 5 Batch 60\n",
      "Epoch 5 Batch 61\n",
      "Epoch 5 Batch 62\n",
      "Epoch 5 Batch 63\n",
      "Epoch 5 Batch 64\n",
      "Epoch 5 Batch 65\n",
      "Epoch 5 Batch 66\n",
      "Epoch 5 Batch 67\n",
      "Epoch 5 Batch 68\n",
      "Epoch 5 Batch 69\n",
      "Epoch 5 Batch 70\n",
      "Epoch 5 Batch 71\n",
      "Epoch 5 Batch 72\n",
      "Epoch 5 Batch 73\n",
      "Epoch 5 Batch 74\n",
      "Epoch 5 Batch 75\n",
      "Epoch 5 Batch 76\n",
      "Epoch 5 Batch 77\n",
      "Epoch 5 Batch 78\n",
      "Epoch 5 Batch 79\n",
      "Epoch 5 Batch 80\n",
      "Epoch 5 Batch 81\n",
      "Epoch 5 Batch 82\n",
      "Epoch 5 Batch 83\n",
      "Epoch 5 Batch 84\n",
      "Epoch 5 Batch 85\n",
      "Epoch 5 Batch 86\n",
      "Epoch 5 Batch 87\n",
      "Epoch 5 Batch 88\n",
      "Epoch 5 Batch 89\n",
      "Epoch 5 Batch 90\n",
      "Epoch 5 Batch 91\n",
      "Epoch 5 Batch 92\n",
      "Epoch 5 Batch 93\n",
      "Epoch 5 Batch 94\n",
      "Epoch 5 Batch 95\n",
      "Epoch 5 Batch 96\n",
      "Epoch 5 Batch 97\n",
      "Epoch 5 Batch 98\n",
      "Epoch 5 Batch 99\n",
      "Epoch 5 Batch 100\n",
      "Epoch 5 Batch 101\n",
      "Epoch 5 Batch 102\n",
      "Epoch 5 Batch 103\n",
      "Epoch 5 Batch 104\n",
      "Epoch 5 Batch 105\n",
      "Epoch 5 Batch 106\n",
      "Epoch 5 Batch 107\n",
      "Epoch 5 Batch 108\n",
      "Epoch 5 Batch 109\n",
      "Epoch 5 Batch 110\n",
      "Epoch 5 Batch 111\n",
      "Epoch 5 Batch 112\n",
      "Epoch 5 Batch 113\n",
      "Epoch 5 Batch 114\n",
      "Epoch 5 Batch 115\n",
      "Epoch 5 Batch 116\n",
      "Epoch 5 Batch 117\n",
      "Epoch 5 Batch 118\n",
      "Epoch 5 Batch 119\n",
      "Epoch 5 Batch 120\n",
      "Epoch 5 Batch 121\n",
      "Epoch 5 Batch 122\n",
      "Epoch 5 Batch 123\n",
      "Epoch 5 Batch 124\n",
      "Epoch 5 Batch 125\n",
      "Epoch 5 Batch 126\n",
      "Epoch 5 Batch 127\n",
      "Epoch 5 Batch 128\n",
      "Epoch 5 Batch 129\n",
      "Epoch 5 Batch 130\n",
      "Epoch 5 Batch 131\n",
      "Epoch 5 Batch 132\n",
      "Epoch 5 Batch 133\n",
      "Epoch 5 Batch 134\n",
      "Epoch 5 Batch 135\n",
      "Epoch 5 Batch 136\n",
      "Epoch 5 Batch 137\n",
      "Epoch 5 Batch 138\n",
      "Epoch 5 Batch 139\n",
      "Epoch 5 Batch 140\n",
      "Epoch 5 Batch 141\n",
      "Epoch 5 Batch 142\n",
      "Epoch 5 Batch 143\n",
      "Epoch 5 Batch 144\n",
      "Epoch 5 Batch 145\n",
      "Epoch 5 Batch 146\n",
      "Epoch 5 Batch 147\n",
      "Epoch 5 Batch 148\n",
      "Epoch 5 Batch 149\n",
      "Epoch 5 Batch 150\n",
      "Epoch 5 Batch 151\n",
      "Epoch 5 Batch 152\n",
      "Epoch 5 Batch 153\n",
      "Epoch 5 Batch 154\n",
      "Epoch 5 Batch 155\n",
      "Epoch 5 Batch 156\n",
      "Epoch 5 Batch 157\n",
      "Epoch 5 Batch 158\n",
      "Epoch 5 Batch 159\n",
      "Epoch 5 Batch 160\n",
      "Epoch 5 Batch 161\n",
      "Epoch 5 Batch 162\n",
      "Epoch 5 Batch 163\n",
      "Epoch 5 Batch 164\n",
      "Epoch 5 Batch 165\n",
      "Epoch 5 Batch 166\n",
      "Epoch 5 Batch 167\n",
      "Epoch 5 Batch 168\n",
      "Epoch 5 Batch 169\n",
      "Epoch 5 Batch 170\n",
      "Epoch 5 Batch 171\n",
      "Epoch 5 Batch 172\n",
      "Epoch 5 Batch 173\n",
      "Epoch 5 Batch 174\n",
      "Epoch 5 Batch 175\n",
      "Epoch 5 Batch 176\n",
      "Epoch 5 Batch 177\n",
      "Epoch 5 Batch 178\n",
      "Epoch 5 Batch 179\n",
      "Epoch 5 Batch 180\n",
      "Epoch 5 Batch 181\n",
      "Epoch 5 Batch 182\n",
      "Epoch 5 Batch 183\n",
      "Epoch 5 Batch 184\n",
      "Epoch 5 Batch 185\n",
      "Epoch 5 Batch 186\n",
      "Epoch 5 Batch 187\n",
      "Epoch 5 Batch 188\n",
      "Epoch 5 Batch 189\n",
      "Epoch 5 Batch 190\n",
      "Epoch 5 Batch 191\n",
      "Epoch 5 Batch 192\n",
      "Epoch 5 Batch 193\n",
      "Epoch 5 Batch 194\n",
      "Epoch 5 Batch 195\n",
      "Epoch 5 Batch 196\n",
      "Epoch 5 Batch 197\n",
      "Epoch 5 Batch 198\n",
      "Epoch 5 Batch 199\n",
      "Epoch 5 Batch 200\n",
      "Epoch 5 Batch 201\n",
      "Epoch 5 Batch 202\n",
      "Epoch 5 Batch 203\n",
      "Epoch 5 Batch 204\n",
      "Epoch 5 Batch 205\n",
      "Epoch 5 Batch 206\n",
      "Epoch 5 Batch 207\n",
      "Epoch 5 Batch 208\n",
      "Epoch 5 Batch 209\n",
      "Epoch 5 Batch 210\n",
      "Epoch 5 Batch 211\n",
      "Epoch 5 Batch 212\n",
      "Epoch 5 Batch 213\n",
      "Epoch 5 Batch 214\n",
      "Epoch 5 Batch 215\n",
      "Epoch 5 Batch 216\n",
      "Epoch 5 Batch 217\n",
      "Epoch 5 Batch 218\n",
      "Epoch 5 Batch 219\n",
      "Epoch 5 Batch 220\n",
      "Epoch 5 Batch 221\n",
      "Epoch 5 Batch 222\n",
      "Epoch 5 Batch 223\n",
      "Epoch 5 Batch 224\n",
      "Epoch 5 Batch 225\n",
      "Epoch 5 Batch 226\n",
      "Epoch 5 Batch 227\n",
      "Epoch 5 Batch 228\n",
      "Epoch 5 Batch 229\n",
      "Epoch 5 Batch 230\n",
      "Epoch 5 Batch 231\n",
      "Epoch 5 Batch 232\n",
      "Epoch 5 Batch 233\n",
      "Epoch 5 Batch 234\n",
      "Epoch 5 Batch 235\n",
      "Epoch 5 Batch 236\n",
      "Epoch 5 Batch 237\n",
      "Epoch 5 Batch 238\n",
      "Epoch 5 Batch 239\n",
      "Epoch 5 Batch 240\n",
      "Epoch 5 Batch 241\n",
      "Epoch 5 Batch 242\n",
      "Epoch 5 Batch 243\n",
      "Epoch 5 Batch 244\n",
      "Epoch 5 Batch 245\n",
      "Epoch 5 Batch 246\n",
      "Epoch 5 Batch 247\n",
      "Epoch 5 Batch 248\n",
      "Epoch 5 Batch 249\n",
      "Epoch 5 Batch 250\n",
      "Epoch 5 Batch 251\n",
      "Epoch 5 Batch 252\n",
      "Epoch 5 Batch 253\n",
      "Epoch 5 Batch 254\n",
      "Epoch 5 Batch 255\n",
      "Epoch 5 Batch 256\n",
      "Epoch 5 Batch 257\n",
      "Epoch 5 Batch 258\n",
      "Epoch 5 Batch 259\n",
      "Epoch 5 Batch 260\n",
      "Epoch 5 Batch 261\n",
      "Epoch 5 Batch 262\n",
      "Epoch 5 Batch 263\n",
      "Epoch 5 Batch 264\n",
      "Epoch 5 Batch 265\n",
      "Epoch 5 Batch 266\n",
      "Epoch 5 Batch 267\n",
      "Epoch 5 Batch 268\n",
      "Epoch 5 Batch 269\n",
      "Epoch 5 Batch 270\n",
      "Epoch 5 Batch 271\n",
      "Epoch 5 Batch 272\n",
      "Epoch 5 Batch 273\n",
      "Epoch 5 Batch 274\n",
      "Epoch 5 Batch 275\n",
      "Epoch 5 Batch 276\n",
      "Epoch 5 Batch 277\n",
      "Epoch 5 Batch 278\n",
      "Epoch 5 Batch 279\n",
      "Epoch 5 Batch 280\n",
      "Epoch 5 Batch 281\n",
      "Epoch 5 Batch 282\n",
      "Epoch 5 Batch 283\n",
      "Epoch 5 Batch 284\n",
      "Epoch 5 Batch 285\n",
      "Epoch 5 Batch 286\n",
      "Epoch 5 Batch 287\n",
      "Epoch 5 Batch 288\n",
      "Epoch 5 Batch 289\n",
      "Epoch 5 Batch 290\n",
      "Epoch 5 Batch 291\n",
      "Epoch 5 Batch 292\n",
      "Epoch 5 Batch 293\n",
      "Epoch 5 Batch 294\n",
      "Epoch 5 Batch 295\n",
      "Epoch 5 Batch 296\n",
      "Epoch 5 Batch 297\n",
      "Epoch 5 Batch 298\n",
      "Epoch 5 Batch 299\n",
      "Epoch 5 Batch 300\n",
      "Epoch 5 Batch 301\n",
      "Epoch 5 Batch 302\n",
      "Epoch 5 Batch 303\n",
      "Epoch 5 Batch 304\n",
      "Epoch 5 Batch 305\n",
      "Epoch 5 Batch 306\n",
      "Epoch 5 Batch 307\n",
      "Epoch 5 Batch 308\n",
      "Epoch 5 Batch 309\n",
      "Epoch 5 Batch 310\n",
      "Epoch 5 Batch 311\n",
      "Epoch 5 Batch 312\n",
      "Epoch 5 Batch 313\n",
      "Epoch 5 Batch 314\n",
      "Epoch 5 Batch 315\n",
      "Epoch 5 Batch 316\n",
      "Epoch 5 Batch 317\n",
      "Epoch 5 Batch 318\n",
      "Epoch 5 Batch 319\n",
      "Epoch 5 Batch 320\n",
      "Epoch 5 Batch 321\n",
      "Epoch 5 Batch 322\n",
      "Epoch 5 Batch 323\n",
      "Epoch 5 Batch 324\n",
      "Epoch 5 Batch 325\n",
      "Epoch 5 Batch 326\n",
      "Epoch 5 Batch 327\n",
      "Epoch 5 Batch 328\n",
      "Epoch 5 Batch 329\n",
      "Epoch 5 Batch 330\n",
      "Epoch 5 Batch 331\n",
      "Epoch 5 Batch 332\n",
      "Epoch 5 Batch 333\n",
      "Epoch 5 Batch 334\n",
      "Epoch 5 Batch 335\n",
      "Epoch 5 Batch 336\n",
      "Epoch 5 Batch 337\n",
      "Epoch 5 Batch 338\n",
      "Epoch 5 Batch 339\n",
      "Epoch 5 Batch 340\n",
      "Epoch 5 Batch 341\n",
      "Epoch 5 Batch 342\n",
      "Epoch 5 Batch 343\n",
      "Epoch 5 Batch 344\n",
      "Epoch 5 Batch 345\n",
      "Epoch 5 Batch 346\n",
      "Epoch 5 Batch 347\n",
      "Epoch 5 Batch 348\n",
      "Epoch 5 Batch 349\n",
      "Epoch 5 Batch 350\n",
      "Epoch 5 Batch 351\n",
      "Epoch 5 Batch 352\n",
      "Epoch 5 Batch 353\n",
      "Epoch 5 Batch 354\n",
      "Epoch 5 Batch 355\n",
      "Epoch 5 Batch 356\n",
      "Epoch 5 Batch 357\n",
      "Epoch 5 Batch 358\n",
      "Epoch 5 Batch 359\n",
      "Epoch 5 Batch 360\n",
      "Epoch 5 Batch 361\n",
      "Epoch 5 Batch 362\n",
      "Epoch 5 Batch 363\n",
      "Epoch 5 Batch 364\n",
      "Epoch 5 Batch 365\n",
      "Epoch 5 Batch 366\n",
      "Epoch 5 Batch 367\n",
      "Epoch 5 Batch 368\n",
      "Epoch 5 Batch 369\n",
      "Epoch 5 Batch 370\n",
      "Epoch 5 Batch 371\n",
      "Epoch 5 Batch 372\n",
      "Epoch 5 Batch 373\n",
      "Epoch 5 Batch 374\n",
      "Epoch 5 Batch 375\n",
      "Epoch 5 Batch 376\n",
      "Epoch 5 Batch 377\n",
      "Epoch 5 Batch 378\n",
      "Epoch 5 Batch 379\n",
      "Epoch 5 Batch 380\n",
      "Epoch 5 Batch 381\n",
      "Epoch 5 Batch 382\n",
      "Epoch 5 Batch 383\n",
      "Epoch 5 Batch 384\n",
      "Epoch 5 Batch 385\n",
      "Epoch 5 Batch 386\n",
      "Epoch 5 Batch 387\n",
      "Epoch 5 Batch 388\n",
      "Epoch 5 Batch 389\n",
      "Epoch 5 Batch 390\n",
      "Done - the last line above this should be: Epoch 5, Batch 390\n"
     ]
    }
   ],
   "source": [
    "from tensorflow.keras.datasets import cifar10\n",
    "import random\n",
    "\n",
    "# Let's use the CIFAR-10 dataset\n",
    "(x_train, y_train), (x_test, y_test) = cifar10.load_data()\n",
    "\n",
    "# We will use a mini-batch size of 128\n",
    "batch_size = 128\n",
    "\n",
    "# Calculate the total number of mini-batches in an epoch\n",
    "# HINT: It has something to do with the (mini) batch size\n",
    "batches = len(x_train) // 128\n",
    "\n",
    "# Let's use a seed so we can randomly shuffle both the pixel data and labels the same shuffle.\n",
    "seed = 101\n",
    "\n",
    "# Let's do 5 passes (epochs) over the dataset\n",
    "epochs = 5\n",
    "for epoch in range(epochs):\n",
    "    # Shuffle the dataset at the beginning of each epoch\n",
    "    # HINT: We have to shuffle the image data and labels from the training data\n",
    "    random.seed(seed)\n",
    "    random.shuffle(x_train)\n",
    "    random.seed(seed)\n",
    "    random.shuffle(y_train)\n",
    "    # Set a new seed for the next shuffle\n",
    "    seed += random.randint(0, 100)\n",
    "    \n",
    "    # Iterate (sequential) through the shuffled training data, one batch at a time.\n",
    "    for batch in range(batches):\n",
    "        # Get the next batch of data\n",
    "        # HINT: if the begin of the batch is at location X, then the end is X + batc\n",
    "        x_batch = x_train[batch * batch_size:(batch+1) * batch_size]\n",
    "        y_batch = y_train[batch * batch_size:(batch+1) * batch_size]\n",
    "        print(\"Epoch\", epoch+1, \"Batch\", batch+1)\n",
    "        \n",
    "print(\"Done - the last line above this should be: Epoch 5, Batch 390\")\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## ImageDataGenerator and Batch Generation\n",
    "\n",
    "In this section, we will use the **Keras** ImageDataGenerator to automatically generate out mini-batches (vs. hand generating them), and shuffling the training data on each epoch."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1 Batch 1\n",
      "Epoch 1 Batch 2\n",
      "Epoch 1 Batch 3\n",
      "Epoch 1 Batch 4\n",
      "Epoch 1 Batch 5\n",
      "Epoch 1 Batch 6\n",
      "Epoch 1 Batch 7\n",
      "Epoch 1 Batch 8\n",
      "Epoch 1 Batch 9\n",
      "Epoch 1 Batch 10\n",
      "Epoch 1 Batch 11\n",
      "Epoch 1 Batch 12\n",
      "Epoch 1 Batch 13\n",
      "Epoch 1 Batch 14\n",
      "Epoch 1 Batch 15\n",
      "Epoch 1 Batch 16\n",
      "Epoch 1 Batch 17\n",
      "Epoch 1 Batch 18\n",
      "Epoch 1 Batch 19\n",
      "Epoch 1 Batch 20\n",
      "Epoch 1 Batch 21\n",
      "Epoch 1 Batch 22\n",
      "Epoch 1 Batch 23\n",
      "Epoch 1 Batch 24\n",
      "Epoch 1 Batch 25\n",
      "Epoch 1 Batch 26\n",
      "Epoch 1 Batch 27\n",
      "Epoch 1 Batch 28\n",
      "Epoch 1 Batch 29\n",
      "Epoch 1 Batch 30\n",
      "Epoch 1 Batch 31\n",
      "Epoch 1 Batch 32\n",
      "Epoch 1 Batch 33\n",
      "Epoch 1 Batch 34\n",
      "Epoch 1 Batch 35\n",
      "Epoch 1 Batch 36\n",
      "Epoch 1 Batch 37\n",
      "Epoch 1 Batch 38\n",
      "Epoch 1 Batch 39\n",
      "Epoch 1 Batch 40\n",
      "Epoch 1 Batch 41\n",
      "Epoch 1 Batch 42\n",
      "Epoch 1 Batch 43\n",
      "Epoch 1 Batch 44\n",
      "Epoch 1 Batch 45\n",
      "Epoch 1 Batch 46\n",
      "Epoch 1 Batch 47\n",
      "Epoch 1 Batch 48\n",
      "Epoch 1 Batch 49\n",
      "Epoch 1 Batch 50\n",
      "Epoch 1 Batch 51\n",
      "Epoch 1 Batch 52\n",
      "Epoch 1 Batch 53\n",
      "Epoch 1 Batch 54\n",
      "Epoch 1 Batch 55\n",
      "Epoch 1 Batch 56\n",
      "Epoch 1 Batch 57\n",
      "Epoch 1 Batch 58\n",
      "Epoch 1 Batch 59\n",
      "Epoch 1 Batch 60\n",
      "Epoch 1 Batch 61\n",
      "Epoch 1 Batch 62\n",
      "Epoch 1 Batch 63\n",
      "Epoch 1 Batch 64\n",
      "Epoch 1 Batch 65\n",
      "Epoch 1 Batch 66\n",
      "Epoch 1 Batch 67\n",
      "Epoch 1 Batch 68\n",
      "Epoch 1 Batch 69\n",
      "Epoch 1 Batch 70\n",
      "Epoch 1 Batch 71\n",
      "Epoch 1 Batch 72\n",
      "Epoch 1 Batch 73\n",
      "Epoch 1 Batch 74\n",
      "Epoch 1 Batch 75\n",
      "Epoch 1 Batch 76\n",
      "Epoch 1 Batch 77\n",
      "Epoch 1 Batch 78\n",
      "Epoch 1 Batch 79\n",
      "Epoch 1 Batch 80\n",
      "Epoch 1 Batch 81\n",
      "Epoch 1 Batch 82\n",
      "Epoch 1 Batch 83\n",
      "Epoch 1 Batch 84\n",
      "Epoch 1 Batch 85\n",
      "Epoch 1 Batch 86\n",
      "Epoch 1 Batch 87\n",
      "Epoch 1 Batch 88\n",
      "Epoch 1 Batch 89\n",
      "Epoch 1 Batch 90\n",
      "Epoch 1 Batch 91\n",
      "Epoch 1 Batch 92\n",
      "Epoch 1 Batch 93\n",
      "Epoch 1 Batch 94\n",
      "Epoch 1 Batch 95\n",
      "Epoch 1 Batch 96\n",
      "Epoch 1 Batch 97\n",
      "Epoch 1 Batch 98\n",
      "Epoch 1 Batch 99\n",
      "Epoch 1 Batch 100\n",
      "Epoch 1 Batch 101\n",
      "Epoch 1 Batch 102\n",
      "Epoch 1 Batch 103\n",
      "Epoch 1 Batch 104\n",
      "Epoch 1 Batch 105\n",
      "Epoch 1 Batch 106\n",
      "Epoch 1 Batch 107\n",
      "Epoch 1 Batch 108\n",
      "Epoch 1 Batch 109\n",
      "Epoch 1 Batch 110\n",
      "Epoch 1 Batch 111\n",
      "Epoch 1 Batch 112\n",
      "Epoch 1 Batch 113\n",
      "Epoch 1 Batch 114\n",
      "Epoch 1 Batch 115\n",
      "Epoch 1 Batch 116\n",
      "Epoch 1 Batch 117\n",
      "Epoch 1 Batch 118\n",
      "Epoch 1 Batch 119\n",
      "Epoch 1 Batch 120\n",
      "Epoch 1 Batch 121\n",
      "Epoch 1 Batch 122\n",
      "Epoch 1 Batch 123\n",
      "Epoch 1 Batch 124\n",
      "Epoch 1 Batch 125\n",
      "Epoch 1 Batch 126\n",
      "Epoch 1 Batch 127\n",
      "Epoch 1 Batch 128\n",
      "Epoch 1 Batch 129\n",
      "Epoch 1 Batch 130\n",
      "Epoch 1 Batch 131\n",
      "Epoch 1 Batch 132\n",
      "Epoch 1 Batch 133\n",
      "Epoch 1 Batch 134\n",
      "Epoch 1 Batch 135\n",
      "Epoch 1 Batch 136\n",
      "Epoch 1 Batch 137\n",
      "Epoch 1 Batch 138\n",
      "Epoch 1 Batch 139\n",
      "Epoch 1 Batch 140\n",
      "Epoch 1 Batch 141\n",
      "Epoch 1 Batch 142\n",
      "Epoch 1 Batch 143\n",
      "Epoch 1 Batch 144\n",
      "Epoch 1 Batch 145\n",
      "Epoch 1 Batch 146\n",
      "Epoch 1 Batch 147\n",
      "Epoch 1 Batch 148\n",
      "Epoch 1 Batch 149\n",
      "Epoch 1 Batch 150\n",
      "Epoch 1 Batch 151\n",
      "Epoch 1 Batch 152\n",
      "Epoch 1 Batch 153\n",
      "Epoch 1 Batch 154\n",
      "Epoch 1 Batch 155\n",
      "Epoch 1 Batch 156\n",
      "Epoch 1 Batch 157\n",
      "Epoch 1 Batch 158\n",
      "Epoch 1 Batch 159\n",
      "Epoch 1 Batch 160\n",
      "Epoch 1 Batch 161\n",
      "Epoch 1 Batch 162\n",
      "Epoch 1 Batch 163\n",
      "Epoch 1 Batch 164\n",
      "Epoch 1 Batch 165\n",
      "Epoch 1 Batch 166\n",
      "Epoch 1 Batch 167\n",
      "Epoch 1 Batch 168\n",
      "Epoch 1 Batch 169\n",
      "Epoch 1 Batch 170\n",
      "Epoch 1 Batch 171\n",
      "Epoch 1 Batch 172\n",
      "Epoch 1 Batch 173\n",
      "Epoch 1 Batch 174\n",
      "Epoch 1 Batch 175\n",
      "Epoch 1 Batch 176\n",
      "Epoch 1 Batch 177\n",
      "Epoch 1 Batch 178\n",
      "Epoch 1 Batch 179\n",
      "Epoch 1 Batch 180\n",
      "Epoch 1 Batch 181\n",
      "Epoch 1 Batch 182\n",
      "Epoch 1 Batch 183\n",
      "Epoch 1 Batch 184\n",
      "Epoch 1 Batch 185\n",
      "Epoch 1 Batch 186\n",
      "Epoch 1 Batch 187\n",
      "Epoch 1 Batch 188\n",
      "Epoch 1 Batch 189\n",
      "Epoch 1 Batch 190\n",
      "Epoch 1 Batch 191\n",
      "Epoch 1 Batch 192\n",
      "Epoch 1 Batch 193\n",
      "Epoch 1 Batch 194\n",
      "Epoch 1 Batch 195\n",
      "Epoch 1 Batch 196\n",
      "Epoch 1 Batch 197\n",
      "Epoch 1 Batch 198\n",
      "Epoch 1 Batch 199\n",
      "Epoch 1 Batch 200\n",
      "Epoch 1 Batch 201\n",
      "Epoch 1 Batch 202\n",
      "Epoch 1 Batch 203\n",
      "Epoch 1 Batch 204\n",
      "Epoch 1 Batch 205\n",
      "Epoch 1 Batch 206\n",
      "Epoch 1 Batch 207\n",
      "Epoch 1 Batch 208\n",
      "Epoch 1 Batch 209\n",
      "Epoch 1 Batch 210\n",
      "Epoch 1 Batch 211\n",
      "Epoch 1 Batch 212\n",
      "Epoch 1 Batch 213\n",
      "Epoch 1 Batch 214\n",
      "Epoch 1 Batch 215\n",
      "Epoch 1 Batch 216\n",
      "Epoch 1 Batch 217\n",
      "Epoch 1 Batch 218\n",
      "Epoch 1 Batch 219\n",
      "Epoch 1 Batch 220\n",
      "Epoch 1 Batch 221\n",
      "Epoch 1 Batch 222\n",
      "Epoch 1 Batch 223\n",
      "Epoch 1 Batch 224\n",
      "Epoch 1 Batch 225\n",
      "Epoch 1 Batch 226\n",
      "Epoch 1 Batch 227\n",
      "Epoch 1 Batch 228\n",
      "Epoch 1 Batch 229\n",
      "Epoch 1 Batch 230\n",
      "Epoch 1 Batch 231\n",
      "Epoch 1 Batch 232\n",
      "Epoch 1 Batch 233\n",
      "Epoch 1 Batch 234\n",
      "Epoch 1 Batch 235\n",
      "Epoch 1 Batch 236\n",
      "Epoch 1 Batch 237\n",
      "Epoch 1 Batch 238\n",
      "Epoch 1 Batch 239\n",
      "Epoch 1 Batch 240\n",
      "Epoch 1 Batch 241\n",
      "Epoch 1 Batch 242\n",
      "Epoch 1 Batch 243\n",
      "Epoch 1 Batch 244\n",
      "Epoch 1 Batch 245\n",
      "Epoch 1 Batch 246\n",
      "Epoch 1 Batch 247\n",
      "Epoch 1 Batch 248\n",
      "Epoch 1 Batch 249\n",
      "Epoch 1 Batch 250\n",
      "Epoch 1 Batch 251\n",
      "Epoch 1 Batch 252\n",
      "Epoch 1 Batch 253\n",
      "Epoch 1 Batch 254\n",
      "Epoch 1 Batch 255\n",
      "Epoch 1 Batch 256\n",
      "Epoch 1 Batch 257\n",
      "Epoch 1 Batch 258\n",
      "Epoch 1 Batch 259\n",
      "Epoch 1 Batch 260\n",
      "Epoch 1 Batch 261\n",
      "Epoch 1 Batch 262\n",
      "Epoch 1 Batch 263\n",
      "Epoch 1 Batch 264\n",
      "Epoch 1 Batch 265\n",
      "Epoch 1 Batch 266\n",
      "Epoch 1 Batch 267\n",
      "Epoch 1 Batch 268\n",
      "Epoch 1 Batch 269\n",
      "Epoch 1 Batch 270\n",
      "Epoch 1 Batch 271\n",
      "Epoch 1 Batch 272\n",
      "Epoch 1 Batch 273\n",
      "Epoch 1 Batch 274\n",
      "Epoch 1 Batch 275\n",
      "Epoch 1 Batch 276\n",
      "Epoch 1 Batch 277\n",
      "Epoch 1 Batch 278\n",
      "Epoch 1 Batch 279\n",
      "Epoch 1 Batch 280\n",
      "Epoch 1 Batch 281\n",
      "Epoch 1 Batch 282\n",
      "Epoch 1 Batch 283\n",
      "Epoch 1 Batch 284\n",
      "Epoch 1 Batch 285\n",
      "Epoch 1 Batch 286\n",
      "Epoch 1 Batch 287\n",
      "Epoch 1 Batch 288\n",
      "Epoch 1 Batch 289\n",
      "Epoch 1 Batch 290\n",
      "Epoch 1 Batch 291\n",
      "Epoch 1 Batch 292\n",
      "Epoch 1 Batch 293\n",
      "Epoch 1 Batch 294\n",
      "Epoch 1 Batch 295\n",
      "Epoch 1 Batch 296\n",
      "Epoch 1 Batch 297\n",
      "Epoch 1 Batch 298\n",
      "Epoch 1 Batch 299\n",
      "Epoch 1 Batch 300\n",
      "Epoch 1 Batch 301\n",
      "Epoch 1 Batch 302\n",
      "Epoch 1 Batch 303\n",
      "Epoch 1 Batch 304\n",
      "Epoch 1 Batch 305\n",
      "Epoch 1 Batch 306\n",
      "Epoch 1 Batch 307\n",
      "Epoch 1 Batch 308\n",
      "Epoch 1 Batch 309\n",
      "Epoch 1 Batch 310\n",
      "Epoch 1 Batch 311\n",
      "Epoch 1 Batch 312\n",
      "Epoch 1 Batch 313\n",
      "Epoch 1 Batch 314\n",
      "Epoch 1 Batch 315\n",
      "Epoch 1 Batch 316\n",
      "Epoch 1 Batch 317\n",
      "Epoch 1 Batch 318\n",
      "Epoch 1 Batch 319\n",
      "Epoch 1 Batch 320\n",
      "Epoch 1 Batch 321\n",
      "Epoch 1 Batch 322\n",
      "Epoch 1 Batch 323\n",
      "Epoch 1 Batch 324\n",
      "Epoch 1 Batch 325\n",
      "Epoch 1 Batch 326\n",
      "Epoch 1 Batch 327\n",
      "Epoch 1 Batch 328\n",
      "Epoch 1 Batch 329\n",
      "Epoch 1 Batch 330\n",
      "Epoch 1 Batch 331\n",
      "Epoch 1 Batch 332\n",
      "Epoch 1 Batch 333\n",
      "Epoch 1 Batch 334\n",
      "Epoch 1 Batch 335\n",
      "Epoch 1 Batch 336\n",
      "Epoch 1 Batch 337\n",
      "Epoch 1 Batch 338\n",
      "Epoch 1 Batch 339\n",
      "Epoch 1 Batch 340\n",
      "Epoch 1 Batch 341\n",
      "Epoch 1 Batch 342\n",
      "Epoch 1 Batch 343\n",
      "Epoch 1 Batch 344\n",
      "Epoch 1 Batch 345\n",
      "Epoch 1 Batch 346\n",
      "Epoch 1 Batch 347\n",
      "Epoch 1 Batch 348\n",
      "Epoch 1 Batch 349\n",
      "Epoch 1 Batch 350\n",
      "Epoch 1 Batch 351\n",
      "Epoch 1 Batch 352\n",
      "Epoch 1 Batch 353\n",
      "Epoch 1 Batch 354\n",
      "Epoch 1 Batch 355\n",
      "Epoch 1 Batch 356\n",
      "Epoch 1 Batch 357\n",
      "Epoch 1 Batch 358\n",
      "Epoch 1 Batch 359\n",
      "Epoch 1 Batch 360\n",
      "Epoch 1 Batch 361\n",
      "Epoch 1 Batch 362\n",
      "Epoch 1 Batch 363\n",
      "Epoch 1 Batch 364\n",
      "Epoch 1 Batch 365\n",
      "Epoch 1 Batch 366\n",
      "Epoch 1 Batch 367\n",
      "Epoch 1 Batch 368\n",
      "Epoch 1 Batch 369\n",
      "Epoch 1 Batch 370\n",
      "Epoch 1 Batch 371\n",
      "Epoch 1 Batch 372\n",
      "Epoch 1 Batch 373\n",
      "Epoch 1 Batch 374\n",
      "Epoch 1 Batch 375\n",
      "Epoch 1 Batch 376\n",
      "Epoch 1 Batch 377\n",
      "Epoch 1 Batch 378\n",
      "Epoch 1 Batch 379\n",
      "Epoch 1 Batch 380\n",
      "Epoch 1 Batch 381\n",
      "Epoch 1 Batch 382\n",
      "Epoch 1 Batch 383\n",
      "Epoch 1 Batch 384\n",
      "Epoch 1 Batch 385\n",
      "Epoch 1 Batch 386\n",
      "Epoch 1 Batch 387\n",
      "Epoch 1 Batch 388\n",
      "Epoch 1 Batch 389\n",
      "Epoch 1 Batch 390\n",
      "Epoch 2 Batch 1\n",
      "Epoch 2 Batch 2\n",
      "Epoch 2 Batch 3\n",
      "Epoch 2 Batch 4\n",
      "Epoch 2 Batch 5\n",
      "Epoch 2 Batch 6\n",
      "Epoch 2 Batch 7\n",
      "Epoch 2 Batch 8\n",
      "Epoch 2 Batch 9\n",
      "Epoch 2 Batch 10\n",
      "Epoch 2 Batch 11\n",
      "Epoch 2 Batch 12\n",
      "Epoch 2 Batch 13\n",
      "Epoch 2 Batch 14\n",
      "Epoch 2 Batch 15\n",
      "Epoch 2 Batch 16\n",
      "Epoch 2 Batch 17\n",
      "Epoch 2 Batch 18\n",
      "Epoch 2 Batch 19\n",
      "Epoch 2 Batch 20\n",
      "Epoch 2 Batch 21\n",
      "Epoch 2 Batch 22\n",
      "Epoch 2 Batch 23\n",
      "Epoch 2 Batch 24\n",
      "Epoch 2 Batch 25\n",
      "Epoch 2 Batch 26\n",
      "Epoch 2 Batch 27\n",
      "Epoch 2 Batch 28\n",
      "Epoch 2 Batch 29\n",
      "Epoch 2 Batch 30\n",
      "Epoch 2 Batch 31\n",
      "Epoch 2 Batch 32\n",
      "Epoch 2 Batch 33\n",
      "Epoch 2 Batch 34\n",
      "Epoch 2 Batch 35\n",
      "Epoch 2 Batch 36\n",
      "Epoch 2 Batch 37\n",
      "Epoch 2 Batch 38\n",
      "Epoch 2 Batch 39\n",
      "Epoch 2 Batch 40\n",
      "Epoch 2 Batch 41\n",
      "Epoch 2 Batch 42\n",
      "Epoch 2 Batch 43\n",
      "Epoch 2 Batch 44\n",
      "Epoch 2 Batch 45\n",
      "Epoch 2 Batch 46\n",
      "Epoch 2 Batch 47\n",
      "Epoch 2 Batch 48\n",
      "Epoch 2 Batch 49\n",
      "Epoch 2 Batch 50\n",
      "Epoch 2 Batch 51\n",
      "Epoch 2 Batch 52\n",
      "Epoch 2 Batch 53\n",
      "Epoch 2 Batch 54\n",
      "Epoch 2 Batch 55\n",
      "Epoch 2 Batch 56\n",
      "Epoch 2 Batch 57\n",
      "Epoch 2 Batch 58\n",
      "Epoch 2 Batch 59\n",
      "Epoch 2 Batch 60\n",
      "Epoch 2 Batch 61\n",
      "Epoch 2 Batch 62\n",
      "Epoch 2 Batch 63\n",
      "Epoch 2 Batch 64\n",
      "Epoch 2 Batch 65\n",
      "Epoch 2 Batch 66\n",
      "Epoch 2 Batch 67\n",
      "Epoch 2 Batch 68\n",
      "Epoch 2 Batch 69\n",
      "Epoch 2 Batch 70\n",
      "Epoch 2 Batch 71\n",
      "Epoch 2 Batch 72\n",
      "Epoch 2 Batch 73\n",
      "Epoch 2 Batch 74\n",
      "Epoch 2 Batch 75\n",
      "Epoch 2 Batch 76\n",
      "Epoch 2 Batch 77\n",
      "Epoch 2 Batch 78\n",
      "Epoch 2 Batch 79\n",
      "Epoch 2 Batch 80\n",
      "Epoch 2 Batch 81\n",
      "Epoch 2 Batch 82\n",
      "Epoch 2 Batch 83\n",
      "Epoch 2 Batch 84\n",
      "Epoch 2 Batch 85\n",
      "Epoch 2 Batch 86\n",
      "Epoch 2 Batch 87\n",
      "Epoch 2 Batch 88\n",
      "Epoch 2 Batch 89\n",
      "Epoch 2 Batch 90\n",
      "Epoch 2 Batch 91\n",
      "Epoch 2 Batch 92\n",
      "Epoch 2 Batch 93\n",
      "Epoch 2 Batch 94\n",
      "Epoch 2 Batch 95\n",
      "Epoch 2 Batch 96\n",
      "Epoch 2 Batch 97\n",
      "Epoch 2 Batch 98\n",
      "Epoch 2 Batch 99\n",
      "Epoch 2 Batch 100\n",
      "Epoch 2 Batch 101\n",
      "Epoch 2 Batch 102\n",
      "Epoch 2 Batch 103\n",
      "Epoch 2 Batch 104\n",
      "Epoch 2 Batch 105\n",
      "Epoch 2 Batch 106\n",
      "Epoch 2 Batch 107\n",
      "Epoch 2 Batch 108\n",
      "Epoch 2 Batch 109\n",
      "Epoch 2 Batch 110\n",
      "Epoch 2 Batch 111\n",
      "Epoch 2 Batch 112\n",
      "Epoch 2 Batch 113\n",
      "Epoch 2 Batch 114\n",
      "Epoch 2 Batch 115\n",
      "Epoch 2 Batch 116\n",
      "Epoch 2 Batch 117\n",
      "Epoch 2 Batch 118\n",
      "Epoch 2 Batch 119\n",
      "Epoch 2 Batch 120\n",
      "Epoch 2 Batch 121\n",
      "Epoch 2 Batch 122\n",
      "Epoch 2 Batch 123\n",
      "Epoch 2 Batch 124\n",
      "Epoch 2 Batch 125\n",
      "Epoch 2 Batch 126\n",
      "Epoch 2 Batch 127\n",
      "Epoch 2 Batch 128\n",
      "Epoch 2 Batch 129\n",
      "Epoch 2 Batch 130\n",
      "Epoch 2 Batch 131\n",
      "Epoch 2 Batch 132\n",
      "Epoch 2 Batch 133\n",
      "Epoch 2 Batch 134\n",
      "Epoch 2 Batch 135\n",
      "Epoch 2 Batch 136\n",
      "Epoch 2 Batch 137\n",
      "Epoch 2 Batch 138\n",
      "Epoch 2 Batch 139\n",
      "Epoch 2 Batch 140\n",
      "Epoch 2 Batch 141\n",
      "Epoch 2 Batch 142\n",
      "Epoch 2 Batch 143\n",
      "Epoch 2 Batch 144\n",
      "Epoch 2 Batch 145\n",
      "Epoch 2 Batch 146\n",
      "Epoch 2 Batch 147\n",
      "Epoch 2 Batch 148\n",
      "Epoch 2 Batch 149\n",
      "Epoch 2 Batch 150\n",
      "Epoch 2 Batch 151\n",
      "Epoch 2 Batch 152\n",
      "Epoch 2 Batch 153\n",
      "Epoch 2 Batch 154\n",
      "Epoch 2 Batch 155\n",
      "Epoch 2 Batch 156\n",
      "Epoch 2 Batch 157\n",
      "Epoch 2 Batch 158\n",
      "Epoch 2 Batch 159\n",
      "Epoch 2 Batch 160\n",
      "Epoch 2 Batch 161\n",
      "Epoch 2 Batch 162\n",
      "Epoch 2 Batch 163\n",
      "Epoch 2 Batch 164\n",
      "Epoch 2 Batch 165\n",
      "Epoch 2 Batch 166\n",
      "Epoch 2 Batch 167\n",
      "Epoch 2 Batch 168\n",
      "Epoch 2 Batch 169\n",
      "Epoch 2 Batch 170\n",
      "Epoch 2 Batch 171\n",
      "Epoch 2 Batch 172\n",
      "Epoch 2 Batch 173\n",
      "Epoch 2 Batch 174\n",
      "Epoch 2 Batch 175\n",
      "Epoch 2 Batch 176\n",
      "Epoch 2 Batch 177\n",
      "Epoch 2 Batch 178\n",
      "Epoch 2 Batch 179\n",
      "Epoch 2 Batch 180\n",
      "Epoch 2 Batch 181\n",
      "Epoch 2 Batch 182\n",
      "Epoch 2 Batch 183\n",
      "Epoch 2 Batch 184\n",
      "Epoch 2 Batch 185\n",
      "Epoch 2 Batch 186\n",
      "Epoch 2 Batch 187\n",
      "Epoch 2 Batch 188\n",
      "Epoch 2 Batch 189\n",
      "Epoch 2 Batch 190\n",
      "Epoch 2 Batch 191\n",
      "Epoch 2 Batch 192\n",
      "Epoch 2 Batch 193\n",
      "Epoch 2 Batch 194\n",
      "Epoch 2 Batch 195\n",
      "Epoch 2 Batch 196\n",
      "Epoch 2 Batch 197\n",
      "Epoch 2 Batch 198\n",
      "Epoch 2 Batch 199\n",
      "Epoch 2 Batch 200\n",
      "Epoch 2 Batch 201\n",
      "Epoch 2 Batch 202\n",
      "Epoch 2 Batch 203\n",
      "Epoch 2 Batch 204\n",
      "Epoch 2 Batch 205\n",
      "Epoch 2 Batch 206\n",
      "Epoch 2 Batch 207\n",
      "Epoch 2 Batch 208\n",
      "Epoch 2 Batch 209\n",
      "Epoch 2 Batch 210\n",
      "Epoch 2 Batch 211\n",
      "Epoch 2 Batch 212\n",
      "Epoch 2 Batch 213\n",
      "Epoch 2 Batch 214\n",
      "Epoch 2 Batch 215\n",
      "Epoch 2 Batch 216\n",
      "Epoch 2 Batch 217\n",
      "Epoch 2 Batch 218\n",
      "Epoch 2 Batch 219\n",
      "Epoch 2 Batch 220\n",
      "Epoch 2 Batch 221\n",
      "Epoch 2 Batch 222\n",
      "Epoch 2 Batch 223\n",
      "Epoch 2 Batch 224\n",
      "Epoch 2 Batch 225\n",
      "Epoch 2 Batch 226\n",
      "Epoch 2 Batch 227\n",
      "Epoch 2 Batch 228\n",
      "Epoch 2 Batch 229\n",
      "Epoch 2 Batch 230\n",
      "Epoch 2 Batch 231\n",
      "Epoch 2 Batch 232\n",
      "Epoch 2 Batch 233\n",
      "Epoch 2 Batch 234\n",
      "Epoch 2 Batch 235\n",
      "Epoch 2 Batch 236\n",
      "Epoch 2 Batch 237\n",
      "Epoch 2 Batch 238\n",
      "Epoch 2 Batch 239\n",
      "Epoch 2 Batch 240\n",
      "Epoch 2 Batch 241\n",
      "Epoch 2 Batch 242\n",
      "Epoch 2 Batch 243\n",
      "Epoch 2 Batch 244\n",
      "Epoch 2 Batch 245\n",
      "Epoch 2 Batch 246\n",
      "Epoch 2 Batch 247\n",
      "Epoch 2 Batch 248\n",
      "Epoch 2 Batch 249\n",
      "Epoch 2 Batch 250\n",
      "Epoch 2 Batch 251\n",
      "Epoch 2 Batch 252\n",
      "Epoch 2 Batch 253\n",
      "Epoch 2 Batch 254\n",
      "Epoch 2 Batch 255\n",
      "Epoch 2 Batch 256\n",
      "Epoch 2 Batch 257\n",
      "Epoch 2 Batch 258\n",
      "Epoch 2 Batch 259\n",
      "Epoch 2 Batch 260\n",
      "Epoch 2 Batch 261\n",
      "Epoch 2 Batch 262\n",
      "Epoch 2 Batch 263\n",
      "Epoch 2 Batch 264\n",
      "Epoch 2 Batch 265\n",
      "Epoch 2 Batch 266\n",
      "Epoch 2 Batch 267\n",
      "Epoch 2 Batch 268\n",
      "Epoch 2 Batch 269\n",
      "Epoch 2 Batch 270\n",
      "Epoch 2 Batch 271\n",
      "Epoch 2 Batch 272\n",
      "Epoch 2 Batch 273\n",
      "Epoch 2 Batch 274\n",
      "Epoch 2 Batch 275\n",
      "Epoch 2 Batch 276\n",
      "Epoch 2 Batch 277\n",
      "Epoch 2 Batch 278\n",
      "Epoch 2 Batch 279\n",
      "Epoch 2 Batch 280\n",
      "Epoch 2 Batch 281\n",
      "Epoch 2 Batch 282\n",
      "Epoch 2 Batch 283\n",
      "Epoch 2 Batch 284\n",
      "Epoch 2 Batch 285\n",
      "Epoch 2 Batch 286\n",
      "Epoch 2 Batch 287\n",
      "Epoch 2 Batch 288\n",
      "Epoch 2 Batch 289\n",
      "Epoch 2 Batch 290\n",
      "Epoch 2 Batch 291\n",
      "Epoch 2 Batch 292\n",
      "Epoch 2 Batch 293\n",
      "Epoch 2 Batch 294\n",
      "Epoch 2 Batch 295\n",
      "Epoch 2 Batch 296\n",
      "Epoch 2 Batch 297\n",
      "Epoch 2 Batch 298\n",
      "Epoch 2 Batch 299\n",
      "Epoch 2 Batch 300\n",
      "Epoch 2 Batch 301\n",
      "Epoch 2 Batch 302\n",
      "Epoch 2 Batch 303\n",
      "Epoch 2 Batch 304\n",
      "Epoch 2 Batch 305\n",
      "Epoch 2 Batch 306\n",
      "Epoch 2 Batch 307\n",
      "Epoch 2 Batch 308\n",
      "Epoch 2 Batch 309\n",
      "Epoch 2 Batch 310\n",
      "Epoch 2 Batch 311\n",
      "Epoch 2 Batch 312\n",
      "Epoch 2 Batch 313\n",
      "Epoch 2 Batch 314\n",
      "Epoch 2 Batch 315\n",
      "Epoch 2 Batch 316\n",
      "Epoch 2 Batch 317\n",
      "Epoch 2 Batch 318\n",
      "Epoch 2 Batch 319\n",
      "Epoch 2 Batch 320\n",
      "Epoch 2 Batch 321\n",
      "Epoch 2 Batch 322\n",
      "Epoch 2 Batch 323\n",
      "Epoch 2 Batch 324\n",
      "Epoch 2 Batch 325\n",
      "Epoch 2 Batch 326\n",
      "Epoch 2 Batch 327\n",
      "Epoch 2 Batch 328\n",
      "Epoch 2 Batch 329\n",
      "Epoch 2 Batch 330\n",
      "Epoch 2 Batch 331\n",
      "Epoch 2 Batch 332\n",
      "Epoch 2 Batch 333\n",
      "Epoch 2 Batch 334\n",
      "Epoch 2 Batch 335\n",
      "Epoch 2 Batch 336\n",
      "Epoch 2 Batch 337\n",
      "Epoch 2 Batch 338\n",
      "Epoch 2 Batch 339\n",
      "Epoch 2 Batch 340\n",
      "Epoch 2 Batch 341\n",
      "Epoch 2 Batch 342\n",
      "Epoch 2 Batch 343\n",
      "Epoch 2 Batch 344\n",
      "Epoch 2 Batch 345\n",
      "Epoch 2 Batch 346\n",
      "Epoch 2 Batch 347\n",
      "Epoch 2 Batch 348\n",
      "Epoch 2 Batch 349\n",
      "Epoch 2 Batch 350\n",
      "Epoch 2 Batch 351\n",
      "Epoch 2 Batch 352\n",
      "Epoch 2 Batch 353\n",
      "Epoch 2 Batch 354\n",
      "Epoch 2 Batch 355\n",
      "Epoch 2 Batch 356\n",
      "Epoch 2 Batch 357\n",
      "Epoch 2 Batch 358\n",
      "Epoch 2 Batch 359\n",
      "Epoch 2 Batch 360\n",
      "Epoch 2 Batch 361\n",
      "Epoch 2 Batch 362\n",
      "Epoch 2 Batch 363\n",
      "Epoch 2 Batch 364\n",
      "Epoch 2 Batch 365\n",
      "Epoch 2 Batch 366\n",
      "Epoch 2 Batch 367\n",
      "Epoch 2 Batch 368\n",
      "Epoch 2 Batch 369\n",
      "Epoch 2 Batch 370\n",
      "Epoch 2 Batch 371\n",
      "Epoch 2 Batch 372\n",
      "Epoch 2 Batch 373\n",
      "Epoch 2 Batch 374\n",
      "Epoch 2 Batch 375\n",
      "Epoch 2 Batch 376\n",
      "Epoch 2 Batch 377\n",
      "Epoch 2 Batch 378\n",
      "Epoch 2 Batch 379\n",
      "Epoch 2 Batch 380\n",
      "Epoch 2 Batch 381\n",
      "Epoch 2 Batch 382\n",
      "Epoch 2 Batch 383\n",
      "Epoch 2 Batch 384\n",
      "Epoch 2 Batch 385\n",
      "Epoch 2 Batch 386\n",
      "Epoch 2 Batch 387\n",
      "Epoch 2 Batch 388\n",
      "Epoch 2 Batch 389\n",
      "Epoch 2 Batch 390\n",
      "Epoch 3 Batch 1\n",
      "Epoch 3 Batch 2\n",
      "Epoch 3 Batch 3\n",
      "Epoch 3 Batch 4\n",
      "Epoch 3 Batch 5\n",
      "Epoch 3 Batch 6\n",
      "Epoch 3 Batch 7\n",
      "Epoch 3 Batch 8\n",
      "Epoch 3 Batch 9\n",
      "Epoch 3 Batch 10\n",
      "Epoch 3 Batch 11\n",
      "Epoch 3 Batch 12\n",
      "Epoch 3 Batch 13\n",
      "Epoch 3 Batch 14\n",
      "Epoch 3 Batch 15\n",
      "Epoch 3 Batch 16\n",
      "Epoch 3 Batch 17\n",
      "Epoch 3 Batch 18\n",
      "Epoch 3 Batch 19\n",
      "Epoch 3 Batch 20\n",
      "Epoch 3 Batch 21\n",
      "Epoch 3 Batch 22\n",
      "Epoch 3 Batch 23\n",
      "Epoch 3 Batch 24\n",
      "Epoch 3 Batch 25\n",
      "Epoch 3 Batch 26\n",
      "Epoch 3 Batch 27\n",
      "Epoch 3 Batch 28\n",
      "Epoch 3 Batch 29\n",
      "Epoch 3 Batch 30\n",
      "Epoch 3 Batch 31\n",
      "Epoch 3 Batch 32\n",
      "Epoch 3 Batch 33\n",
      "Epoch 3 Batch 34\n",
      "Epoch 3 Batch 35\n",
      "Epoch 3 Batch 36\n",
      "Epoch 3 Batch 37\n",
      "Epoch 3 Batch 38\n",
      "Epoch 3 Batch 39\n",
      "Epoch 3 Batch 40\n",
      "Epoch 3 Batch 41\n",
      "Epoch 3 Batch 42\n",
      "Epoch 3 Batch 43\n",
      "Epoch 3 Batch 44\n",
      "Epoch 3 Batch 45\n",
      "Epoch 3 Batch 46\n",
      "Epoch 3 Batch 47\n",
      "Epoch 3 Batch 48\n",
      "Epoch 3 Batch 49\n",
      "Epoch 3 Batch 50\n",
      "Epoch 3 Batch 51\n",
      "Epoch 3 Batch 52\n",
      "Epoch 3 Batch 53\n",
      "Epoch 3 Batch 54\n",
      "Epoch 3 Batch 55\n",
      "Epoch 3 Batch 56\n",
      "Epoch 3 Batch 57\n",
      "Epoch 3 Batch 58\n",
      "Epoch 3 Batch 59\n",
      "Epoch 3 Batch 60\n",
      "Epoch 3 Batch 61\n",
      "Epoch 3 Batch 62\n",
      "Epoch 3 Batch 63\n",
      "Epoch 3 Batch 64\n",
      "Epoch 3 Batch 65\n",
      "Epoch 3 Batch 66\n",
      "Epoch 3 Batch 67\n",
      "Epoch 3 Batch 68\n",
      "Epoch 3 Batch 69\n",
      "Epoch 3 Batch 70\n",
      "Epoch 3 Batch 71\n",
      "Epoch 3 Batch 72\n",
      "Epoch 3 Batch 73\n",
      "Epoch 3 Batch 74\n",
      "Epoch 3 Batch 75\n",
      "Epoch 3 Batch 76\n",
      "Epoch 3 Batch 77\n",
      "Epoch 3 Batch 78\n",
      "Epoch 3 Batch 79\n",
      "Epoch 3 Batch 80\n",
      "Epoch 3 Batch 81\n",
      "Epoch 3 Batch 82\n",
      "Epoch 3 Batch 83\n",
      "Epoch 3 Batch 84\n",
      "Epoch 3 Batch 85\n",
      "Epoch 3 Batch 86\n",
      "Epoch 3 Batch 87\n",
      "Epoch 3 Batch 88\n",
      "Epoch 3 Batch 89\n",
      "Epoch 3 Batch 90\n",
      "Epoch 3 Batch 91\n",
      "Epoch 3 Batch 92\n",
      "Epoch 3 Batch 93\n",
      "Epoch 3 Batch 94\n",
      "Epoch 3 Batch 95\n",
      "Epoch 3 Batch 96\n",
      "Epoch 3 Batch 97\n",
      "Epoch 3 Batch 98\n",
      "Epoch 3 Batch 99\n",
      "Epoch 3 Batch 100\n",
      "Epoch 3 Batch 101\n",
      "Epoch 3 Batch 102\n",
      "Epoch 3 Batch 103\n",
      "Epoch 3 Batch 104\n",
      "Epoch 3 Batch 105\n",
      "Epoch 3 Batch 106\n",
      "Epoch 3 Batch 107\n",
      "Epoch 3 Batch 108\n",
      "Epoch 3 Batch 109\n",
      "Epoch 3 Batch 110\n",
      "Epoch 3 Batch 111\n",
      "Epoch 3 Batch 112\n",
      "Epoch 3 Batch 113\n",
      "Epoch 3 Batch 114\n",
      "Epoch 3 Batch 115\n",
      "Epoch 3 Batch 116\n",
      "Epoch 3 Batch 117\n",
      "Epoch 3 Batch 118\n",
      "Epoch 3 Batch 119\n",
      "Epoch 3 Batch 120\n",
      "Epoch 3 Batch 121\n",
      "Epoch 3 Batch 122\n",
      "Epoch 3 Batch 123\n",
      "Epoch 3 Batch 124\n",
      "Epoch 3 Batch 125\n",
      "Epoch 3 Batch 126\n",
      "Epoch 3 Batch 127\n",
      "Epoch 3 Batch 128\n",
      "Epoch 3 Batch 129\n",
      "Epoch 3 Batch 130\n",
      "Epoch 3 Batch 131\n",
      "Epoch 3 Batch 132\n",
      "Epoch 3 Batch 133\n",
      "Epoch 3 Batch 134\n",
      "Epoch 3 Batch 135\n",
      "Epoch 3 Batch 136\n",
      "Epoch 3 Batch 137\n",
      "Epoch 3 Batch 138\n",
      "Epoch 3 Batch 139\n",
      "Epoch 3 Batch 140\n",
      "Epoch 3 Batch 141\n",
      "Epoch 3 Batch 142\n",
      "Epoch 3 Batch 143\n",
      "Epoch 3 Batch 144\n",
      "Epoch 3 Batch 145\n",
      "Epoch 3 Batch 146\n",
      "Epoch 3 Batch 147\n",
      "Epoch 3 Batch 148\n",
      "Epoch 3 Batch 149\n",
      "Epoch 3 Batch 150\n",
      "Epoch 3 Batch 151\n",
      "Epoch 3 Batch 152\n",
      "Epoch 3 Batch 153\n",
      "Epoch 3 Batch 154\n",
      "Epoch 3 Batch 155\n",
      "Epoch 3 Batch 156\n",
      "Epoch 3 Batch 157\n",
      "Epoch 3 Batch 158\n",
      "Epoch 3 Batch 159\n",
      "Epoch 3 Batch 160\n",
      "Epoch 3 Batch 161\n",
      "Epoch 3 Batch 162\n",
      "Epoch 3 Batch 163\n",
      "Epoch 3 Batch 164\n",
      "Epoch 3 Batch 165\n",
      "Epoch 3 Batch 166\n",
      "Epoch 3 Batch 167\n",
      "Epoch 3 Batch 168\n",
      "Epoch 3 Batch 169\n",
      "Epoch 3 Batch 170\n",
      "Epoch 3 Batch 171\n",
      "Epoch 3 Batch 172\n",
      "Epoch 3 Batch 173\n",
      "Epoch 3 Batch 174\n",
      "Epoch 3 Batch 175\n",
      "Epoch 3 Batch 176\n",
      "Epoch 3 Batch 177\n",
      "Epoch 3 Batch 178\n",
      "Epoch 3 Batch 179\n",
      "Epoch 3 Batch 180\n",
      "Epoch 3 Batch 181\n",
      "Epoch 3 Batch 182\n",
      "Epoch 3 Batch 183\n",
      "Epoch 3 Batch 184\n",
      "Epoch 3 Batch 185\n",
      "Epoch 3 Batch 186\n",
      "Epoch 3 Batch 187\n",
      "Epoch 3 Batch 188\n",
      "Epoch 3 Batch 189\n",
      "Epoch 3 Batch 190\n",
      "Epoch 3 Batch 191\n",
      "Epoch 3 Batch 192\n",
      "Epoch 3 Batch 193\n",
      "Epoch 3 Batch 194\n",
      "Epoch 3 Batch 195\n",
      "Epoch 3 Batch 196\n",
      "Epoch 3 Batch 197\n",
      "Epoch 3 Batch 198\n",
      "Epoch 3 Batch 199\n",
      "Epoch 3 Batch 200\n",
      "Epoch 3 Batch 201\n",
      "Epoch 3 Batch 202\n",
      "Epoch 3 Batch 203\n",
      "Epoch 3 Batch 204\n",
      "Epoch 3 Batch 205\n",
      "Epoch 3 Batch 206\n",
      "Epoch 3 Batch 207\n",
      "Epoch 3 Batch 208\n",
      "Epoch 3 Batch 209\n",
      "Epoch 3 Batch 210\n",
      "Epoch 3 Batch 211\n",
      "Epoch 3 Batch 212\n",
      "Epoch 3 Batch 213\n",
      "Epoch 3 Batch 214\n",
      "Epoch 3 Batch 215\n",
      "Epoch 3 Batch 216\n",
      "Epoch 3 Batch 217\n",
      "Epoch 3 Batch 218\n",
      "Epoch 3 Batch 219\n",
      "Epoch 3 Batch 220\n",
      "Epoch 3 Batch 221\n",
      "Epoch 3 Batch 222\n",
      "Epoch 3 Batch 223\n",
      "Epoch 3 Batch 224\n",
      "Epoch 3 Batch 225\n",
      "Epoch 3 Batch 226\n",
      "Epoch 3 Batch 227\n",
      "Epoch 3 Batch 228\n",
      "Epoch 3 Batch 229\n",
      "Epoch 3 Batch 230\n",
      "Epoch 3 Batch 231\n",
      "Epoch 3 Batch 232\n",
      "Epoch 3 Batch 233\n",
      "Epoch 3 Batch 234\n",
      "Epoch 3 Batch 235\n",
      "Epoch 3 Batch 236\n",
      "Epoch 3 Batch 237\n",
      "Epoch 3 Batch 238\n",
      "Epoch 3 Batch 239\n",
      "Epoch 3 Batch 240\n",
      "Epoch 3 Batch 241\n",
      "Epoch 3 Batch 242\n",
      "Epoch 3 Batch 243\n",
      "Epoch 3 Batch 244\n",
      "Epoch 3 Batch 245\n",
      "Epoch 3 Batch 246\n",
      "Epoch 3 Batch 247\n",
      "Epoch 3 Batch 248\n",
      "Epoch 3 Batch 249\n",
      "Epoch 3 Batch 250\n",
      "Epoch 3 Batch 251\n",
      "Epoch 3 Batch 252\n",
      "Epoch 3 Batch 253\n",
      "Epoch 3 Batch 254\n",
      "Epoch 3 Batch 255\n",
      "Epoch 3 Batch 256\n",
      "Epoch 3 Batch 257\n",
      "Epoch 3 Batch 258\n",
      "Epoch 3 Batch 259\n",
      "Epoch 3 Batch 260\n",
      "Epoch 3 Batch 261\n",
      "Epoch 3 Batch 262\n",
      "Epoch 3 Batch 263\n",
      "Epoch 3 Batch 264\n",
      "Epoch 3 Batch 265\n",
      "Epoch 3 Batch 266\n",
      "Epoch 3 Batch 267\n",
      "Epoch 3 Batch 268\n",
      "Epoch 3 Batch 269\n",
      "Epoch 3 Batch 270\n",
      "Epoch 3 Batch 271\n",
      "Epoch 3 Batch 272\n",
      "Epoch 3 Batch 273\n",
      "Epoch 3 Batch 274\n",
      "Epoch 3 Batch 275\n",
      "Epoch 3 Batch 276\n",
      "Epoch 3 Batch 277\n",
      "Epoch 3 Batch 278\n",
      "Epoch 3 Batch 279\n",
      "Epoch 3 Batch 280\n",
      "Epoch 3 Batch 281\n",
      "Epoch 3 Batch 282\n",
      "Epoch 3 Batch 283\n",
      "Epoch 3 Batch 284\n",
      "Epoch 3 Batch 285\n",
      "Epoch 3 Batch 286\n",
      "Epoch 3 Batch 287\n",
      "Epoch 3 Batch 288\n",
      "Epoch 3 Batch 289\n",
      "Epoch 3 Batch 290\n",
      "Epoch 3 Batch 291\n",
      "Epoch 3 Batch 292\n",
      "Epoch 3 Batch 293\n",
      "Epoch 3 Batch 294\n",
      "Epoch 3 Batch 295\n",
      "Epoch 3 Batch 296\n",
      "Epoch 3 Batch 297\n",
      "Epoch 3 Batch 298\n",
      "Epoch 3 Batch 299\n",
      "Epoch 3 Batch 300\n",
      "Epoch 3 Batch 301\n",
      "Epoch 3 Batch 302\n",
      "Epoch 3 Batch 303\n",
      "Epoch 3 Batch 304\n",
      "Epoch 3 Batch 305\n",
      "Epoch 3 Batch 306\n",
      "Epoch 3 Batch 307\n",
      "Epoch 3 Batch 308\n",
      "Epoch 3 Batch 309\n",
      "Epoch 3 Batch 310\n",
      "Epoch 3 Batch 311\n",
      "Epoch 3 Batch 312\n",
      "Epoch 3 Batch 313\n",
      "Epoch 3 Batch 314\n",
      "Epoch 3 Batch 315\n",
      "Epoch 3 Batch 316\n",
      "Epoch 3 Batch 317\n",
      "Epoch 3 Batch 318\n",
      "Epoch 3 Batch 319\n",
      "Epoch 3 Batch 320\n",
      "Epoch 3 Batch 321\n",
      "Epoch 3 Batch 322\n",
      "Epoch 3 Batch 323\n",
      "Epoch 3 Batch 324\n",
      "Epoch 3 Batch 325\n",
      "Epoch 3 Batch 326\n",
      "Epoch 3 Batch 327\n",
      "Epoch 3 Batch 328\n",
      "Epoch 3 Batch 329\n",
      "Epoch 3 Batch 330\n",
      "Epoch 3 Batch 331\n",
      "Epoch 3 Batch 332\n",
      "Epoch 3 Batch 333\n",
      "Epoch 3 Batch 334\n",
      "Epoch 3 Batch 335\n",
      "Epoch 3 Batch 336\n",
      "Epoch 3 Batch 337\n",
      "Epoch 3 Batch 338\n",
      "Epoch 3 Batch 339\n",
      "Epoch 3 Batch 340\n",
      "Epoch 3 Batch 341\n",
      "Epoch 3 Batch 342\n",
      "Epoch 3 Batch 343\n",
      "Epoch 3 Batch 344\n",
      "Epoch 3 Batch 345\n",
      "Epoch 3 Batch 346\n",
      "Epoch 3 Batch 347\n",
      "Epoch 3 Batch 348\n",
      "Epoch 3 Batch 349\n",
      "Epoch 3 Batch 350\n",
      "Epoch 3 Batch 351\n",
      "Epoch 3 Batch 352\n",
      "Epoch 3 Batch 353\n",
      "Epoch 3 Batch 354\n",
      "Epoch 3 Batch 355\n",
      "Epoch 3 Batch 356\n",
      "Epoch 3 Batch 357\n",
      "Epoch 3 Batch 358\n",
      "Epoch 3 Batch 359\n",
      "Epoch 3 Batch 360\n",
      "Epoch 3 Batch 361\n",
      "Epoch 3 Batch 362\n",
      "Epoch 3 Batch 363\n",
      "Epoch 3 Batch 364\n",
      "Epoch 3 Batch 365\n",
      "Epoch 3 Batch 366\n",
      "Epoch 3 Batch 367\n",
      "Epoch 3 Batch 368\n",
      "Epoch 3 Batch 369\n",
      "Epoch 3 Batch 370\n",
      "Epoch 3 Batch 371\n",
      "Epoch 3 Batch 372\n",
      "Epoch 3 Batch 373\n",
      "Epoch 3 Batch 374\n",
      "Epoch 3 Batch 375\n",
      "Epoch 3 Batch 376\n",
      "Epoch 3 Batch 377\n",
      "Epoch 3 Batch 378\n",
      "Epoch 3 Batch 379\n",
      "Epoch 3 Batch 380\n",
      "Epoch 3 Batch 381\n",
      "Epoch 3 Batch 382\n",
      "Epoch 3 Batch 383\n",
      "Epoch 3 Batch 384\n",
      "Epoch 3 Batch 385\n",
      "Epoch 3 Batch 386\n",
      "Epoch 3 Batch 387\n",
      "Epoch 3 Batch 388\n",
      "Epoch 3 Batch 389\n",
      "Epoch 3 Batch 390\n",
      "Epoch 4 Batch 1\n",
      "Epoch 4 Batch 2\n",
      "Epoch 4 Batch 3\n",
      "Epoch 4 Batch 4\n",
      "Epoch 4 Batch 5\n",
      "Epoch 4 Batch 6\n",
      "Epoch 4 Batch 7\n",
      "Epoch 4 Batch 8\n",
      "Epoch 4 Batch 9\n",
      "Epoch 4 Batch 10\n",
      "Epoch 4 Batch 11\n",
      "Epoch 4 Batch 12\n",
      "Epoch 4 Batch 13\n",
      "Epoch 4 Batch 14\n",
      "Epoch 4 Batch 15\n",
      "Epoch 4 Batch 16\n",
      "Epoch 4 Batch 17\n",
      "Epoch 4 Batch 18\n",
      "Epoch 4 Batch 19\n",
      "Epoch 4 Batch 20\n",
      "Epoch 4 Batch 21\n",
      "Epoch 4 Batch 22\n",
      "Epoch 4 Batch 23\n",
      "Epoch 4 Batch 24\n",
      "Epoch 4 Batch 25\n",
      "Epoch 4 Batch 26\n",
      "Epoch 4 Batch 27\n",
      "Epoch 4 Batch 28\n",
      "Epoch 4 Batch 29\n",
      "Epoch 4 Batch 30\n",
      "Epoch 4 Batch 31\n",
      "Epoch 4 Batch 32\n",
      "Epoch 4 Batch 33\n",
      "Epoch 4 Batch 34\n",
      "Epoch 4 Batch 35\n",
      "Epoch 4 Batch 36\n",
      "Epoch 4 Batch 37\n",
      "Epoch 4 Batch 38\n",
      "Epoch 4 Batch 39\n",
      "Epoch 4 Batch 40\n",
      "Epoch 4 Batch 41\n",
      "Epoch 4 Batch 42\n",
      "Epoch 4 Batch 43\n",
      "Epoch 4 Batch 44\n",
      "Epoch 4 Batch 45\n",
      "Epoch 4 Batch 46\n",
      "Epoch 4 Batch 47\n",
      "Epoch 4 Batch 48\n",
      "Epoch 4 Batch 49\n",
      "Epoch 4 Batch 50\n",
      "Epoch 4 Batch 51\n",
      "Epoch 4 Batch 52\n",
      "Epoch 4 Batch 53\n",
      "Epoch 4 Batch 54\n",
      "Epoch 4 Batch 55\n",
      "Epoch 4 Batch 56\n",
      "Epoch 4 Batch 57\n",
      "Epoch 4 Batch 58\n",
      "Epoch 4 Batch 59\n",
      "Epoch 4 Batch 60\n",
      "Epoch 4 Batch 61\n",
      "Epoch 4 Batch 62\n",
      "Epoch 4 Batch 63\n",
      "Epoch 4 Batch 64\n",
      "Epoch 4 Batch 65\n",
      "Epoch 4 Batch 66\n",
      "Epoch 4 Batch 67\n",
      "Epoch 4 Batch 68\n",
      "Epoch 4 Batch 69\n",
      "Epoch 4 Batch 70\n",
      "Epoch 4 Batch 71\n",
      "Epoch 4 Batch 72\n",
      "Epoch 4 Batch 73\n",
      "Epoch 4 Batch 74\n",
      "Epoch 4 Batch 75\n",
      "Epoch 4 Batch 76\n",
      "Epoch 4 Batch 77\n",
      "Epoch 4 Batch 78\n",
      "Epoch 4 Batch 79\n",
      "Epoch 4 Batch 80\n",
      "Epoch 4 Batch 81\n",
      "Epoch 4 Batch 82\n",
      "Epoch 4 Batch 83\n",
      "Epoch 4 Batch 84\n",
      "Epoch 4 Batch 85\n",
      "Epoch 4 Batch 86\n",
      "Epoch 4 Batch 87\n",
      "Epoch 4 Batch 88\n",
      "Epoch 4 Batch 89\n",
      "Epoch 4 Batch 90\n",
      "Epoch 4 Batch 91\n",
      "Epoch 4 Batch 92\n",
      "Epoch 4 Batch 93\n",
      "Epoch 4 Batch 94\n",
      "Epoch 4 Batch 95\n",
      "Epoch 4 Batch 96\n",
      "Epoch 4 Batch 97\n",
      "Epoch 4 Batch 98\n",
      "Epoch 4 Batch 99\n",
      "Epoch 4 Batch 100\n",
      "Epoch 4 Batch 101\n",
      "Epoch 4 Batch 102\n",
      "Epoch 4 Batch 103\n",
      "Epoch 4 Batch 104\n",
      "Epoch 4 Batch 105\n",
      "Epoch 4 Batch 106\n",
      "Epoch 4 Batch 107\n",
      "Epoch 4 Batch 108\n",
      "Epoch 4 Batch 109\n",
      "Epoch 4 Batch 110\n",
      "Epoch 4 Batch 111\n",
      "Epoch 4 Batch 112\n",
      "Epoch 4 Batch 113\n",
      "Epoch 4 Batch 114\n",
      "Epoch 4 Batch 115\n",
      "Epoch 4 Batch 116\n",
      "Epoch 4 Batch 117\n",
      "Epoch 4 Batch 118\n",
      "Epoch 4 Batch 119\n",
      "Epoch 4 Batch 120\n",
      "Epoch 4 Batch 121\n",
      "Epoch 4 Batch 122\n",
      "Epoch 4 Batch 123\n",
      "Epoch 4 Batch 124\n",
      "Epoch 4 Batch 125\n",
      "Epoch 4 Batch 126\n",
      "Epoch 4 Batch 127\n",
      "Epoch 4 Batch 128\n",
      "Epoch 4 Batch 129\n",
      "Epoch 4 Batch 130\n",
      "Epoch 4 Batch 131\n",
      "Epoch 4 Batch 132\n",
      "Epoch 4 Batch 133\n",
      "Epoch 4 Batch 134\n",
      "Epoch 4 Batch 135\n",
      "Epoch 4 Batch 136\n",
      "Epoch 4 Batch 137\n",
      "Epoch 4 Batch 138\n",
      "Epoch 4 Batch 139\n",
      "Epoch 4 Batch 140\n",
      "Epoch 4 Batch 141\n",
      "Epoch 4 Batch 142\n",
      "Epoch 4 Batch 143\n",
      "Epoch 4 Batch 144\n",
      "Epoch 4 Batch 145\n",
      "Epoch 4 Batch 146\n",
      "Epoch 4 Batch 147\n",
      "Epoch 4 Batch 148\n",
      "Epoch 4 Batch 149\n",
      "Epoch 4 Batch 150\n",
      "Epoch 4 Batch 151\n",
      "Epoch 4 Batch 152\n",
      "Epoch 4 Batch 153\n",
      "Epoch 4 Batch 154\n",
      "Epoch 4 Batch 155\n",
      "Epoch 4 Batch 156\n",
      "Epoch 4 Batch 157\n",
      "Epoch 4 Batch 158\n",
      "Epoch 4 Batch 159\n",
      "Epoch 4 Batch 160\n",
      "Epoch 4 Batch 161\n",
      "Epoch 4 Batch 162\n",
      "Epoch 4 Batch 163\n",
      "Epoch 4 Batch 164\n",
      "Epoch 4 Batch 165\n",
      "Epoch 4 Batch 166\n",
      "Epoch 4 Batch 167\n",
      "Epoch 4 Batch 168\n",
      "Epoch 4 Batch 169\n",
      "Epoch 4 Batch 170\n",
      "Epoch 4 Batch 171\n",
      "Epoch 4 Batch 172\n",
      "Epoch 4 Batch 173\n",
      "Epoch 4 Batch 174\n",
      "Epoch 4 Batch 175\n",
      "Epoch 4 Batch 176\n",
      "Epoch 4 Batch 177\n",
      "Epoch 4 Batch 178\n",
      "Epoch 4 Batch 179\n",
      "Epoch 4 Batch 180\n",
      "Epoch 4 Batch 181\n",
      "Epoch 4 Batch 182\n",
      "Epoch 4 Batch 183\n",
      "Epoch 4 Batch 184\n",
      "Epoch 4 Batch 185\n",
      "Epoch 4 Batch 186\n",
      "Epoch 4 Batch 187\n",
      "Epoch 4 Batch 188\n",
      "Epoch 4 Batch 189\n",
      "Epoch 4 Batch 190\n",
      "Epoch 4 Batch 191\n",
      "Epoch 4 Batch 192\n",
      "Epoch 4 Batch 193\n",
      "Epoch 4 Batch 194\n",
      "Epoch 4 Batch 195\n",
      "Epoch 4 Batch 196\n",
      "Epoch 4 Batch 197\n",
      "Epoch 4 Batch 198\n",
      "Epoch 4 Batch 199\n",
      "Epoch 4 Batch 200\n",
      "Epoch 4 Batch 201\n",
      "Epoch 4 Batch 202\n",
      "Epoch 4 Batch 203\n",
      "Epoch 4 Batch 204\n",
      "Epoch 4 Batch 205\n",
      "Epoch 4 Batch 206\n",
      "Epoch 4 Batch 207\n",
      "Epoch 4 Batch 208\n",
      "Epoch 4 Batch 209\n",
      "Epoch 4 Batch 210\n",
      "Epoch 4 Batch 211\n",
      "Epoch 4 Batch 212\n",
      "Epoch 4 Batch 213\n",
      "Epoch 4 Batch 214\n",
      "Epoch 4 Batch 215\n",
      "Epoch 4 Batch 216\n",
      "Epoch 4 Batch 217\n",
      "Epoch 4 Batch 218\n",
      "Epoch 4 Batch 219\n",
      "Epoch 4 Batch 220\n",
      "Epoch 4 Batch 221\n",
      "Epoch 4 Batch 222\n",
      "Epoch 4 Batch 223\n",
      "Epoch 4 Batch 224\n",
      "Epoch 4 Batch 225\n",
      "Epoch 4 Batch 226\n",
      "Epoch 4 Batch 227\n",
      "Epoch 4 Batch 228\n",
      "Epoch 4 Batch 229\n",
      "Epoch 4 Batch 230\n",
      "Epoch 4 Batch 231\n",
      "Epoch 4 Batch 232\n",
      "Epoch 4 Batch 233\n",
      "Epoch 4 Batch 234\n",
      "Epoch 4 Batch 235\n",
      "Epoch 4 Batch 236\n",
      "Epoch 4 Batch 237\n",
      "Epoch 4 Batch 238\n",
      "Epoch 4 Batch 239\n",
      "Epoch 4 Batch 240\n",
      "Epoch 4 Batch 241\n",
      "Epoch 4 Batch 242\n",
      "Epoch 4 Batch 243\n",
      "Epoch 4 Batch 244\n",
      "Epoch 4 Batch 245\n",
      "Epoch 4 Batch 246\n",
      "Epoch 4 Batch 247\n",
      "Epoch 4 Batch 248\n",
      "Epoch 4 Batch 249\n",
      "Epoch 4 Batch 250\n",
      "Epoch 4 Batch 251\n",
      "Epoch 4 Batch 252\n",
      "Epoch 4 Batch 253\n",
      "Epoch 4 Batch 254\n",
      "Epoch 4 Batch 255\n",
      "Epoch 4 Batch 256\n",
      "Epoch 4 Batch 257\n",
      "Epoch 4 Batch 258\n",
      "Epoch 4 Batch 259\n",
      "Epoch 4 Batch 260\n",
      "Epoch 4 Batch 261\n",
      "Epoch 4 Batch 262\n",
      "Epoch 4 Batch 263\n",
      "Epoch 4 Batch 264\n",
      "Epoch 4 Batch 265\n",
      "Epoch 4 Batch 266\n",
      "Epoch 4 Batch 267\n",
      "Epoch 4 Batch 268\n",
      "Epoch 4 Batch 269\n",
      "Epoch 4 Batch 270\n",
      "Epoch 4 Batch 271\n",
      "Epoch 4 Batch 272\n",
      "Epoch 4 Batch 273\n",
      "Epoch 4 Batch 274\n",
      "Epoch 4 Batch 275\n",
      "Epoch 4 Batch 276\n",
      "Epoch 4 Batch 277\n",
      "Epoch 4 Batch 278\n",
      "Epoch 4 Batch 279\n",
      "Epoch 4 Batch 280\n",
      "Epoch 4 Batch 281\n",
      "Epoch 4 Batch 282\n",
      "Epoch 4 Batch 283\n",
      "Epoch 4 Batch 284\n",
      "Epoch 4 Batch 285\n",
      "Epoch 4 Batch 286\n",
      "Epoch 4 Batch 287\n",
      "Epoch 4 Batch 288\n",
      "Epoch 4 Batch 289\n",
      "Epoch 4 Batch 290\n",
      "Epoch 4 Batch 291\n",
      "Epoch 4 Batch 292\n",
      "Epoch 4 Batch 293\n",
      "Epoch 4 Batch 294\n",
      "Epoch 4 Batch 295\n",
      "Epoch 4 Batch 296\n",
      "Epoch 4 Batch 297\n",
      "Epoch 4 Batch 298\n",
      "Epoch 4 Batch 299\n",
      "Epoch 4 Batch 300\n",
      "Epoch 4 Batch 301\n",
      "Epoch 4 Batch 302\n",
      "Epoch 4 Batch 303\n",
      "Epoch 4 Batch 304\n",
      "Epoch 4 Batch 305\n",
      "Epoch 4 Batch 306\n",
      "Epoch 4 Batch 307\n",
      "Epoch 4 Batch 308\n",
      "Epoch 4 Batch 309\n",
      "Epoch 4 Batch 310\n",
      "Epoch 4 Batch 311\n",
      "Epoch 4 Batch 312\n",
      "Epoch 4 Batch 313\n",
      "Epoch 4 Batch 314\n",
      "Epoch 4 Batch 315\n",
      "Epoch 4 Batch 316\n",
      "Epoch 4 Batch 317\n",
      "Epoch 4 Batch 318\n",
      "Epoch 4 Batch 319\n",
      "Epoch 4 Batch 320\n",
      "Epoch 4 Batch 321\n",
      "Epoch 4 Batch 322\n",
      "Epoch 4 Batch 323\n",
      "Epoch 4 Batch 324\n",
      "Epoch 4 Batch 325\n",
      "Epoch 4 Batch 326\n",
      "Epoch 4 Batch 327\n",
      "Epoch 4 Batch 328\n",
      "Epoch 4 Batch 329\n",
      "Epoch 4 Batch 330\n",
      "Epoch 4 Batch 331\n",
      "Epoch 4 Batch 332\n",
      "Epoch 4 Batch 333\n",
      "Epoch 4 Batch 334\n",
      "Epoch 4 Batch 335\n",
      "Epoch 4 Batch 336\n",
      "Epoch 4 Batch 337\n",
      "Epoch 4 Batch 338\n",
      "Epoch 4 Batch 339\n",
      "Epoch 4 Batch 340\n",
      "Epoch 4 Batch 341\n",
      "Epoch 4 Batch 342\n",
      "Epoch 4 Batch 343\n",
      "Epoch 4 Batch 344\n",
      "Epoch 4 Batch 345\n",
      "Epoch 4 Batch 346\n",
      "Epoch 4 Batch 347\n",
      "Epoch 4 Batch 348\n",
      "Epoch 4 Batch 349\n",
      "Epoch 4 Batch 350\n",
      "Epoch 4 Batch 351\n",
      "Epoch 4 Batch 352\n",
      "Epoch 4 Batch 353\n",
      "Epoch 4 Batch 354\n",
      "Epoch 4 Batch 355\n",
      "Epoch 4 Batch 356\n",
      "Epoch 4 Batch 357\n",
      "Epoch 4 Batch 358\n",
      "Epoch 4 Batch 359\n",
      "Epoch 4 Batch 360\n",
      "Epoch 4 Batch 361\n",
      "Epoch 4 Batch 362\n",
      "Epoch 4 Batch 363\n",
      "Epoch 4 Batch 364\n",
      "Epoch 4 Batch 365\n",
      "Epoch 4 Batch 366\n",
      "Epoch 4 Batch 367\n",
      "Epoch 4 Batch 368\n",
      "Epoch 4 Batch 369\n",
      "Epoch 4 Batch 370\n",
      "Epoch 4 Batch 371\n",
      "Epoch 4 Batch 372\n",
      "Epoch 4 Batch 373\n",
      "Epoch 4 Batch 374\n",
      "Epoch 4 Batch 375\n",
      "Epoch 4 Batch 376\n",
      "Epoch 4 Batch 377\n",
      "Epoch 4 Batch 378\n",
      "Epoch 4 Batch 379\n",
      "Epoch 4 Batch 380\n",
      "Epoch 4 Batch 381\n",
      "Epoch 4 Batch 382\n",
      "Epoch 4 Batch 383\n",
      "Epoch 4 Batch 384\n",
      "Epoch 4 Batch 385\n",
      "Epoch 4 Batch 386\n",
      "Epoch 4 Batch 387\n",
      "Epoch 4 Batch 388\n",
      "Epoch 4 Batch 389\n",
      "Epoch 4 Batch 390\n",
      "Epoch 5 Batch 1\n",
      "Epoch 5 Batch 2\n",
      "Epoch 5 Batch 3\n",
      "Epoch 5 Batch 4\n",
      "Epoch 5 Batch 5\n",
      "Epoch 5 Batch 6\n",
      "Epoch 5 Batch 7\n",
      "Epoch 5 Batch 8\n",
      "Epoch 5 Batch 9\n",
      "Epoch 5 Batch 10\n",
      "Epoch 5 Batch 11\n",
      "Epoch 5 Batch 12\n",
      "Epoch 5 Batch 13\n",
      "Epoch 5 Batch 14\n",
      "Epoch 5 Batch 15\n",
      "Epoch 5 Batch 16\n",
      "Epoch 5 Batch 17\n",
      "Epoch 5 Batch 18\n",
      "Epoch 5 Batch 19\n",
      "Epoch 5 Batch 20\n",
      "Epoch 5 Batch 21\n",
      "Epoch 5 Batch 22\n",
      "Epoch 5 Batch 23\n",
      "Epoch 5 Batch 24\n",
      "Epoch 5 Batch 25\n",
      "Epoch 5 Batch 26\n",
      "Epoch 5 Batch 27\n",
      "Epoch 5 Batch 28\n",
      "Epoch 5 Batch 29\n",
      "Epoch 5 Batch 30\n",
      "Epoch 5 Batch 31\n",
      "Epoch 5 Batch 32\n",
      "Epoch 5 Batch 33\n",
      "Epoch 5 Batch 34\n",
      "Epoch 5 Batch 35\n",
      "Epoch 5 Batch 36\n",
      "Epoch 5 Batch 37\n",
      "Epoch 5 Batch 38\n",
      "Epoch 5 Batch 39\n",
      "Epoch 5 Batch 40\n",
      "Epoch 5 Batch 41\n",
      "Epoch 5 Batch 42\n",
      "Epoch 5 Batch 43\n",
      "Epoch 5 Batch 44\n",
      "Epoch 5 Batch 45\n",
      "Epoch 5 Batch 46\n",
      "Epoch 5 Batch 47\n",
      "Epoch 5 Batch 48\n",
      "Epoch 5 Batch 49\n",
      "Epoch 5 Batch 50\n",
      "Epoch 5 Batch 51\n",
      "Epoch 5 Batch 52\n",
      "Epoch 5 Batch 53\n",
      "Epoch 5 Batch 54\n",
      "Epoch 5 Batch 55\n",
      "Epoch 5 Batch 56\n",
      "Epoch 5 Batch 57\n",
      "Epoch 5 Batch 58\n",
      "Epoch 5 Batch 59\n",
      "Epoch 5 Batch 60\n",
      "Epoch 5 Batch 61\n",
      "Epoch 5 Batch 62\n",
      "Epoch 5 Batch 63\n",
      "Epoch 5 Batch 64\n",
      "Epoch 5 Batch 65\n",
      "Epoch 5 Batch 66\n",
      "Epoch 5 Batch 67\n",
      "Epoch 5 Batch 68\n",
      "Epoch 5 Batch 69\n",
      "Epoch 5 Batch 70\n",
      "Epoch 5 Batch 71\n",
      "Epoch 5 Batch 72\n",
      "Epoch 5 Batch 73\n",
      "Epoch 5 Batch 74\n",
      "Epoch 5 Batch 75\n",
      "Epoch 5 Batch 76\n",
      "Epoch 5 Batch 77\n",
      "Epoch 5 Batch 78\n",
      "Epoch 5 Batch 79\n",
      "Epoch 5 Batch 80\n",
      "Epoch 5 Batch 81\n",
      "Epoch 5 Batch 82\n",
      "Epoch 5 Batch 83\n",
      "Epoch 5 Batch 84\n",
      "Epoch 5 Batch 85\n",
      "Epoch 5 Batch 86\n",
      "Epoch 5 Batch 87\n",
      "Epoch 5 Batch 88\n",
      "Epoch 5 Batch 89\n",
      "Epoch 5 Batch 90\n",
      "Epoch 5 Batch 91\n",
      "Epoch 5 Batch 92\n",
      "Epoch 5 Batch 93\n",
      "Epoch 5 Batch 94\n",
      "Epoch 5 Batch 95\n",
      "Epoch 5 Batch 96\n",
      "Epoch 5 Batch 97\n",
      "Epoch 5 Batch 98\n",
      "Epoch 5 Batch 99\n",
      "Epoch 5 Batch 100\n",
      "Epoch 5 Batch 101\n",
      "Epoch 5 Batch 102\n",
      "Epoch 5 Batch 103\n",
      "Epoch 5 Batch 104\n",
      "Epoch 5 Batch 105\n",
      "Epoch 5 Batch 106\n",
      "Epoch 5 Batch 107\n",
      "Epoch 5 Batch 108\n",
      "Epoch 5 Batch 109\n",
      "Epoch 5 Batch 110\n",
      "Epoch 5 Batch 111\n",
      "Epoch 5 Batch 112\n",
      "Epoch 5 Batch 113\n",
      "Epoch 5 Batch 114\n",
      "Epoch 5 Batch 115\n",
      "Epoch 5 Batch 116\n",
      "Epoch 5 Batch 117\n",
      "Epoch 5 Batch 118\n",
      "Epoch 5 Batch 119\n",
      "Epoch 5 Batch 120\n",
      "Epoch 5 Batch 121\n",
      "Epoch 5 Batch 122\n",
      "Epoch 5 Batch 123\n",
      "Epoch 5 Batch 124\n",
      "Epoch 5 Batch 125\n",
      "Epoch 5 Batch 126\n",
      "Epoch 5 Batch 127\n",
      "Epoch 5 Batch 128\n",
      "Epoch 5 Batch 129\n",
      "Epoch 5 Batch 130\n",
      "Epoch 5 Batch 131\n",
      "Epoch 5 Batch 132\n",
      "Epoch 5 Batch 133\n",
      "Epoch 5 Batch 134\n",
      "Epoch 5 Batch 135\n",
      "Epoch 5 Batch 136\n",
      "Epoch 5 Batch 137\n",
      "Epoch 5 Batch 138\n",
      "Epoch 5 Batch 139\n",
      "Epoch 5 Batch 140\n",
      "Epoch 5 Batch 141\n",
      "Epoch 5 Batch 142\n",
      "Epoch 5 Batch 143\n",
      "Epoch 5 Batch 144\n",
      "Epoch 5 Batch 145\n",
      "Epoch 5 Batch 146\n",
      "Epoch 5 Batch 147\n",
      "Epoch 5 Batch 148\n",
      "Epoch 5 Batch 149\n",
      "Epoch 5 Batch 150\n",
      "Epoch 5 Batch 151\n",
      "Epoch 5 Batch 152\n",
      "Epoch 5 Batch 153\n",
      "Epoch 5 Batch 154\n",
      "Epoch 5 Batch 155\n",
      "Epoch 5 Batch 156\n",
      "Epoch 5 Batch 157\n",
      "Epoch 5 Batch 158\n",
      "Epoch 5 Batch 159\n",
      "Epoch 5 Batch 160\n",
      "Epoch 5 Batch 161\n",
      "Epoch 5 Batch 162\n",
      "Epoch 5 Batch 163\n",
      "Epoch 5 Batch 164\n",
      "Epoch 5 Batch 165\n",
      "Epoch 5 Batch 166\n",
      "Epoch 5 Batch 167\n",
      "Epoch 5 Batch 168\n",
      "Epoch 5 Batch 169\n",
      "Epoch 5 Batch 170\n",
      "Epoch 5 Batch 171\n",
      "Epoch 5 Batch 172\n",
      "Epoch 5 Batch 173\n",
      "Epoch 5 Batch 174\n",
      "Epoch 5 Batch 175\n",
      "Epoch 5 Batch 176\n",
      "Epoch 5 Batch 177\n",
      "Epoch 5 Batch 178\n",
      "Epoch 5 Batch 179\n",
      "Epoch 5 Batch 180\n",
      "Epoch 5 Batch 181\n",
      "Epoch 5 Batch 182\n",
      "Epoch 5 Batch 183\n",
      "Epoch 5 Batch 184\n",
      "Epoch 5 Batch 185\n",
      "Epoch 5 Batch 186\n",
      "Epoch 5 Batch 187\n",
      "Epoch 5 Batch 188\n",
      "Epoch 5 Batch 189\n",
      "Epoch 5 Batch 190\n",
      "Epoch 5 Batch 191\n",
      "Epoch 5 Batch 192\n",
      "Epoch 5 Batch 193\n",
      "Epoch 5 Batch 194\n",
      "Epoch 5 Batch 195\n",
      "Epoch 5 Batch 196\n",
      "Epoch 5 Batch 197\n",
      "Epoch 5 Batch 198\n",
      "Epoch 5 Batch 199\n",
      "Epoch 5 Batch 200\n",
      "Epoch 5 Batch 201\n",
      "Epoch 5 Batch 202\n",
      "Epoch 5 Batch 203\n",
      "Epoch 5 Batch 204\n",
      "Epoch 5 Batch 205\n",
      "Epoch 5 Batch 206\n",
      "Epoch 5 Batch 207\n",
      "Epoch 5 Batch 208\n",
      "Epoch 5 Batch 209\n",
      "Epoch 5 Batch 210\n",
      "Epoch 5 Batch 211\n",
      "Epoch 5 Batch 212\n",
      "Epoch 5 Batch 213\n",
      "Epoch 5 Batch 214\n",
      "Epoch 5 Batch 215\n",
      "Epoch 5 Batch 216\n",
      "Epoch 5 Batch 217\n",
      "Epoch 5 Batch 218\n",
      "Epoch 5 Batch 219\n",
      "Epoch 5 Batch 220\n",
      "Epoch 5 Batch 221\n",
      "Epoch 5 Batch 222\n",
      "Epoch 5 Batch 223\n",
      "Epoch 5 Batch 224\n",
      "Epoch 5 Batch 225\n",
      "Epoch 5 Batch 226\n",
      "Epoch 5 Batch 227\n",
      "Epoch 5 Batch 228\n",
      "Epoch 5 Batch 229\n",
      "Epoch 5 Batch 230\n",
      "Epoch 5 Batch 231\n",
      "Epoch 5 Batch 232\n",
      "Epoch 5 Batch 233\n",
      "Epoch 5 Batch 234\n",
      "Epoch 5 Batch 235\n",
      "Epoch 5 Batch 236\n",
      "Epoch 5 Batch 237\n",
      "Epoch 5 Batch 238\n",
      "Epoch 5 Batch 239\n",
      "Epoch 5 Batch 240\n",
      "Epoch 5 Batch 241\n",
      "Epoch 5 Batch 242\n",
      "Epoch 5 Batch 243\n",
      "Epoch 5 Batch 244\n",
      "Epoch 5 Batch 245\n",
      "Epoch 5 Batch 246\n",
      "Epoch 5 Batch 247\n",
      "Epoch 5 Batch 248\n",
      "Epoch 5 Batch 249\n",
      "Epoch 5 Batch 250\n",
      "Epoch 5 Batch 251\n",
      "Epoch 5 Batch 252\n",
      "Epoch 5 Batch 253\n",
      "Epoch 5 Batch 254\n",
      "Epoch 5 Batch 255\n",
      "Epoch 5 Batch 256\n",
      "Epoch 5 Batch 257\n",
      "Epoch 5 Batch 258\n",
      "Epoch 5 Batch 259\n",
      "Epoch 5 Batch 260\n",
      "Epoch 5 Batch 261\n",
      "Epoch 5 Batch 262\n",
      "Epoch 5 Batch 263\n",
      "Epoch 5 Batch 264\n",
      "Epoch 5 Batch 265\n",
      "Epoch 5 Batch 266\n",
      "Epoch 5 Batch 267\n",
      "Epoch 5 Batch 268\n",
      "Epoch 5 Batch 269\n",
      "Epoch 5 Batch 270\n",
      "Epoch 5 Batch 271\n",
      "Epoch 5 Batch 272\n",
      "Epoch 5 Batch 273\n",
      "Epoch 5 Batch 274\n",
      "Epoch 5 Batch 275\n",
      "Epoch 5 Batch 276\n",
      "Epoch 5 Batch 277\n",
      "Epoch 5 Batch 278\n",
      "Epoch 5 Batch 279\n",
      "Epoch 5 Batch 280\n",
      "Epoch 5 Batch 281\n",
      "Epoch 5 Batch 282\n",
      "Epoch 5 Batch 283\n",
      "Epoch 5 Batch 284\n",
      "Epoch 5 Batch 285\n",
      "Epoch 5 Batch 286\n",
      "Epoch 5 Batch 287\n",
      "Epoch 5 Batch 288\n",
      "Epoch 5 Batch 289\n",
      "Epoch 5 Batch 290\n",
      "Epoch 5 Batch 291\n",
      "Epoch 5 Batch 292\n",
      "Epoch 5 Batch 293\n",
      "Epoch 5 Batch 294\n",
      "Epoch 5 Batch 295\n",
      "Epoch 5 Batch 296\n",
      "Epoch 5 Batch 297\n",
      "Epoch 5 Batch 298\n",
      "Epoch 5 Batch 299\n",
      "Epoch 5 Batch 300\n",
      "Epoch 5 Batch 301\n",
      "Epoch 5 Batch 302\n",
      "Epoch 5 Batch 303\n",
      "Epoch 5 Batch 304\n",
      "Epoch 5 Batch 305\n",
      "Epoch 5 Batch 306\n",
      "Epoch 5 Batch 307\n",
      "Epoch 5 Batch 308\n",
      "Epoch 5 Batch 309\n",
      "Epoch 5 Batch 310\n",
      "Epoch 5 Batch 311\n",
      "Epoch 5 Batch 312\n",
      "Epoch 5 Batch 313\n",
      "Epoch 5 Batch 314\n",
      "Epoch 5 Batch 315\n",
      "Epoch 5 Batch 316\n",
      "Epoch 5 Batch 317\n",
      "Epoch 5 Batch 318\n",
      "Epoch 5 Batch 319\n",
      "Epoch 5 Batch 320\n",
      "Epoch 5 Batch 321\n",
      "Epoch 5 Batch 322\n",
      "Epoch 5 Batch 323\n",
      "Epoch 5 Batch 324\n",
      "Epoch 5 Batch 325\n",
      "Epoch 5 Batch 326\n",
      "Epoch 5 Batch 327\n",
      "Epoch 5 Batch 328\n",
      "Epoch 5 Batch 329\n",
      "Epoch 5 Batch 330\n",
      "Epoch 5 Batch 331\n",
      "Epoch 5 Batch 332\n",
      "Epoch 5 Batch 333\n",
      "Epoch 5 Batch 334\n",
      "Epoch 5 Batch 335\n",
      "Epoch 5 Batch 336\n",
      "Epoch 5 Batch 337\n",
      "Epoch 5 Batch 338\n",
      "Epoch 5 Batch 339\n",
      "Epoch 5 Batch 340\n",
      "Epoch 5 Batch 341\n",
      "Epoch 5 Batch 342\n",
      "Epoch 5 Batch 343\n",
      "Epoch 5 Batch 344\n",
      "Epoch 5 Batch 345\n",
      "Epoch 5 Batch 346\n",
      "Epoch 5 Batch 347\n",
      "Epoch 5 Batch 348\n",
      "Epoch 5 Batch 349\n",
      "Epoch 5 Batch 350\n",
      "Epoch 5 Batch 351\n",
      "Epoch 5 Batch 352\n",
      "Epoch 5 Batch 353\n",
      "Epoch 5 Batch 354\n",
      "Epoch 5 Batch 355\n",
      "Epoch 5 Batch 356\n",
      "Epoch 5 Batch 357\n",
      "Epoch 5 Batch 358\n",
      "Epoch 5 Batch 359\n",
      "Epoch 5 Batch 360\n",
      "Epoch 5 Batch 361\n",
      "Epoch 5 Batch 362\n",
      "Epoch 5 Batch 363\n",
      "Epoch 5 Batch 364\n",
      "Epoch 5 Batch 365\n",
      "Epoch 5 Batch 366\n",
      "Epoch 5 Batch 367\n",
      "Epoch 5 Batch 368\n",
      "Epoch 5 Batch 369\n",
      "Epoch 5 Batch 370\n",
      "Epoch 5 Batch 371\n",
      "Epoch 5 Batch 372\n",
      "Epoch 5 Batch 373\n",
      "Epoch 5 Batch 374\n",
      "Epoch 5 Batch 375\n",
      "Epoch 5 Batch 376\n",
      "Epoch 5 Batch 377\n",
      "Epoch 5 Batch 378\n",
      "Epoch 5 Batch 379\n",
      "Epoch 5 Batch 380\n",
      "Epoch 5 Batch 381\n",
      "Epoch 5 Batch 382\n",
      "Epoch 5 Batch 383\n",
      "Epoch 5 Batch 384\n",
      "Epoch 5 Batch 385\n",
      "Epoch 5 Batch 386\n",
      "Epoch 5 Batch 387\n",
      "Epoch 5 Batch 388\n",
      "Epoch 5 Batch 389\n",
      "Epoch 5 Batch 390\n",
      "Done - the last line above this should be: Epoch 5, Batch 390\n"
     ]
    }
   ],
   "source": [
    "from tensorflow.keras.preprocessing.image import ImageDataGenerator\n",
    "\n",
    "# Let's use the CIFAR-10 dataset\n",
    "(x_train, y_train), (x_test, y_test) = cifar10.load_data()\n",
    "\n",
    "# We will use a mini-batch size of 128\n",
    "batch_size = 128\n",
    "\n",
    "# Calculate the total number of mini-batches in an epoch\n",
    "batches = len(x_train) // batch_size\n",
    "\n",
    "# instantiate an Image Data generator object\n",
    "# HINT: Image Data generator is a big giveaway\n",
    "datagen = ImageDataGenerator()\n",
    "\n",
    "# Let's do 5 passes (epochs) over the dataset\n",
    "epochs = 5\n",
    "for epoch in range(epochs):\n",
    "    \n",
    "    # Use generator to create batches\n",
    "    # HINT: The method is about flowing data from in-memory (vs. on-disk)\n",
    "    batch = 0\n",
    "    for x_batch, y_batch in datagen.flow(x_train, y_train, batch_size=batch_size, shuffle=True):\n",
    "        batch += 1 # Keep track of the number of batches so far.\n",
    "        print(\"Epoch\", epoch+1, \"Batch\", batch)\n",
    "        \n",
    "        # At the end of the training data, let's loop around for the next epoch.\n",
    "        if batch == batches:\n",
    "            break\n",
    "            \n",
    "print(\"Done - the last line above this should be: Epoch 5, Batch 390\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Learning Rate\n",
    "\n",
    "Let's show how to do short epochs to get a feel on what might be the right learning rate for your training."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/3\n",
      "32/32 [==============================] - 0s 8ms/step - loss: 364.0993 - accuracy: 0.1060\n",
      "Epoch 2/3\n",
      "32/32 [==============================] - 0s 4ms/step - loss: 2.3161 - accuracy: 0.1040\n",
      "Epoch 3/3\n",
      "32/32 [==============================] - 0s 4ms/step - loss: 2.3107 - accuracy: 0.0870\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x7f7c21b42bd0>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from tensorflow.keras import Sequential, optimizers\n",
    "from tensorflow.keras.layers import Conv2D, MaxPooling2D, Flatten, Dense\n",
    "from tensorflow.keras.utils import to_categorical\n",
    "import numpy as np\n",
    "\n",
    "# Let's use the CIFAR-10 dataset\n",
    "(x_train, y_train), (x_test, y_test) = cifar10.load_data()\n",
    "\n",
    "# Normalize the pixel data\n",
    "x_train = (x_train / 255.0).astype(np.float32)\n",
    "x_test  = (x_test  / 255.0).astype(np.float32)\n",
    "\n",
    "# One-hot encode the labels\n",
    "y_train = to_categorical(y_train)\n",
    "y_test  = to_categorical(y_test)\n",
    "\n",
    "\n",
    "def convNet(input_shape, nclasses):\n",
    "    model = Sequential()\n",
    "    model.add(Conv2D(32, kernel_size=(3, 3),\n",
    "                     activation='relu',\n",
    "                     input_shape=input_shape))\n",
    "    model.add(Conv2D(64, (3, 3), activation='relu'))\n",
    "    model.add(MaxPooling2D(pool_size=(2, 2)))\n",
    "    model.add(Flatten())\n",
    "    model.add(Dense(128, activation='relu'))\n",
    "    model.add(Dense(nclasses, activation='softmax'))\n",
    "    return model\n",
    "\n",
    "# Create a simple CNN and set learning rate very high (0.1))\n",
    "# HINT: how would you abbreviate learning rate?\n",
    "model = convNet((32, 32, 3), 10)\n",
    "model.compile(loss='categorical_crossentropy',\n",
    "              optimizer=optimizers.Adam(lr=0.1),\n",
    "              metrics=['accuracy'])\n",
    "\n",
    "# Let's take a fraction of the training data to test the learning rate (2%)\n",
    "x_tmp = x_train[0:1000]\n",
    "y_tmp = y_train[0:1000]\n",
    "\n",
    "# Let's run 3 epochs at learning rate = 0.1\n",
    "model.fit(x_tmp, y_tmp, epochs=3, batch_size=32, verbose=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Argh, it's horrible. The loss on the first epoch is high (14.0+) and then never goes down - like it's stuck.\n",
    "\n",
    "Hum, okay now you experiment with different learning rates to find one where the loss goes down rapidly and a steady increase in accuracy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/3\n",
      "32/32 [==============================] - 0s 5ms/step - loss: 2.3176 - accuracy: 0.1360\n",
      "Epoch 2/3\n",
      "32/32 [==============================] - 0s 5ms/step - loss: 2.0158 - accuracy: 0.2730\n",
      "Epoch 3/3\n",
      "32/32 [==============================] - 0s 4ms/step - loss: 1.7211 - accuracy: 0.3970\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x7f7bc0730450>"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model = convNet((32, 32, 3), 10)\n",
    "# Pick your own learning rate until the results are good.\n",
    "# HINT: It's going to be a lot smaller than 0.1\n",
    "model.compile(loss='categorical_crossentropy',\n",
    "              optimizer=optimizers.Adam(lr=0.001),\n",
    "              metrics=['accuracy'])\n",
    "\n",
    "# Let's run 3 epochs at your learning rate\n",
    "model.fit(x_tmp, y_tmp, epochs=3, batch_size=32, verbose=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## End of Lab Exercise"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
